-- Logs begin at Thu 2022-06-23 18:45:28 UTC, end at Thu 2022-06-23 20:28:02 UTC. -- Jun 23 19:47:02.493029 dv-cinder-847274-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 23 19:47:02.504843 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 23 19:47:02.506873 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 23 19:47:02 2022] *** Jun 23 19:47:02.506873 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 23 19:47:02.506873 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 23 19:47:02.506873 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: nodename: dv-cinder-847274-2-iscsi Jun 23 19:47:02.506873 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: machine: x86_64 Jun 23 19:47:02.506873 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: clock source: unix Jun 23 19:47:02.506873 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: pcre jit disabled Jun 23 19:47:02.506873 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: detected number of CPU cores: 4 Jun 23 19:47:02.506873 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: current working directory: / Jun 23 19:47:02.506873 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: detected binary path: /usr/bin/uwsgi-core Jun 23 19:47:02.507515 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: your processes number limit is 47680 Jun 23 19:47:02.507515 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: your memory page size is 4096 bytes Jun 23 19:47:02.507515 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: detected max file descriptor number: 2048 Jun 23 19:47:02.507515 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: lock engine: pthread robust mutexes Jun 23 19:47:02.507515 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: thunder lock: enabled Jun 23 19:47:02.507515 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 23 19:47:02.507515 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 23 19:47:02.551598 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: Python main interpreter initialized at 0x556657f1e9c0 Jun 23 19:47:02.551598 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: python threads support enabled Jun 23 19:47:02.551598 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: your server socket listen backlog is limited to 100 connections Jun 23 19:47:02.551598 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: your mercy for graceful operations on workers is 90 seconds Jun 23 19:47:02.551916 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: mapped 403077 bytes (393 KB) for 2 cores Jun 23 19:47:02.552019 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: *** Operational MODE: preforking *** Jun 23 19:47:02.552284 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: *** uWSGI is running in multiple interpreter mode *** Jun 23 19:47:02.552284 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: spawned uWSGI master process (pid: 80352) Jun 23 19:47:02.552493 dv-cinder-847274-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 23 19:47:02.555119 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: spawned uWSGI worker 1 (pid: 80362, cores: 1) Jun 23 19:47:02.555119 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: spawned uWSGI worker 2 (pid: 80363, cores: 1) Jun 23 19:47:02.555119 dv-cinder-847274-2-iscsi devstack@keystone.service[80352]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 23 19:47:08.005607 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 19:47:08.005607 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: from cryptography.utils import int_from_bytes Jun 23 19:47:08.006435 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 19:47:08.006435 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: from cryptography.utils import int_from_bytes Jun 23 19:47:08.054704 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 23 19:47:08.054896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 23 19:47:08.055090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] command line args: [] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 23 19:47:08.055911 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 23 19:47:08.056091 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ================================================================================ {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 23 19:47:08.056362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] admin_token = **** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.056609 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] config_dir = [] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.056795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.056976 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] config_source = [] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.057155 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] debug = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.057352 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.057705 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.057886 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.057981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 19:47:08.057981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: from cryptography.utils import int_from_bytes Jun 23 19:47:08.058421 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.058486 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 19:47:08.058486 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: from cryptography.utils import int_from_bytes Jun 23 19:47:08.058593 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.058593 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] list_limit = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.058815 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] log_config_append = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.058931 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.059111 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] log_dir = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.059287 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] log_file = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.059525 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.059774 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.060215 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.060437 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.060437 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.060750 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.060927 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.061104 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.061283 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.061509 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.061744 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.061977 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.062241 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.062469 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.062995 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.063477 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.44/identity {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.063642 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] publish_errors = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.063875 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.064105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.064279 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.064550 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.064785 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.065018 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] standard_threads = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.065249 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.065492 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.065669 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.065840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] use_journal = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.066016 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] use_json = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.066191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] use_stderr = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.066367 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] use_syslog = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.066596 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.066851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.067090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.067349 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.067588 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.067837 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.068074 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.068338 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.068582 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.068830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.069078 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.069331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.069565 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.069811 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.070046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.070292 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.070563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.070804 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.071060 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.071313 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.071556 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.071794 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.072047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.072319 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.072577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.072826 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.073057 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.073286 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.073514 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.073748 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.073994 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.074240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.074469 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.074699 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.074944 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.075189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.075426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.075672 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.075902 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.076136 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.076392 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.076626 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.076859 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.077091 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.077323 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.077560 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.077807 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.078039 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.078275 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.078506 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.078741 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.078976 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.079222 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.079457 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.079694 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.079930 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.080166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.080424 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.080662 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.080893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.081139 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.081374 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.081610 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.081842 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.082099 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.082335 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.082566 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.082803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.083039 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.083304 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.083540 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.083789 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.084031 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.084267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.084527 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.084764 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.085010 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.085247 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.085484 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.085726 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.085964 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.086199 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.086476 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.086714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.086951 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.087209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.087449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.087689 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.087928 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.088169 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.088435 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.088674 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.088913 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.089145 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.089381 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.089613 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.089849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.090079 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.090319 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.090555 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.090803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.091042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.091298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.091556 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.091796 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.092028 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] auth.external = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.092258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.092577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.092811 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.093043 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] auth.password = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.093273 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] auth.token = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.093513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.093747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.093991 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.094222 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.094467 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.094715 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.094946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.095180 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] credential.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.095427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.095676 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.096002 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.096154 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.096481 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.096777 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.096980 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.097251 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.097472 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.097725 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.098009 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.098284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.098530 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.098772 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.099039 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.099250 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] federation.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.099487 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.099732 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.099978 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.100208 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.100478 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.100724 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.100980 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.101219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.101474 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.101711 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.101960 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.102722 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.102971 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.103219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.103458 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.103693 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.103940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.104174 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.104434 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.104686 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.104939 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.105173 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.105405 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.105637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.105880 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.106127 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.106375 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.106630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.106877 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.107172 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.107386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.107624 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.107855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.108092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.108341 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.108580 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.108819 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.109057 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.109305 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.109536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.109786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.110032 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.110267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.110511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.110756 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.110985 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.111221 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.111452 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.111689 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.111924 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.112166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.112456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.112660 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.112907 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.113152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.113381 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.113610 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.113857 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.114102 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.114340 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.114584 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.114816 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 23 19:47:08.114948 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.115072 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.115188 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 23 19:47:08.115302 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.115416 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] command line args: [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 23 19:47:08.115537 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.115726 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.115852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 23 19:47:08.115973 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.116095 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ================================================================================ {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 23 19:47:08.116217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.116365 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] admin_token = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.116536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.116680 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.116796 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] config_dir = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.116918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.117034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.117160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.117276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] config_source = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.117390 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.117505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] debug = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.117619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.118382 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.118500 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.118641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.118829 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.118948 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.119067 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.119191 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.119315 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.119541 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.119660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.119776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.119986 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.120116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.120233 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.120450 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] list_limit = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.120570 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.120685 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.120798 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] log_config_append = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.120929 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.121053 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.121170 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.121351 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] log_dir = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.121469 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.121607 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.121733 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] log_file = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.121849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.122061 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.122180 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.122295 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.122499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.122628 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.122744 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.122950 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.123069 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.123185 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.123398 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.123533 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.123650 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.123764 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.123878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.124020 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.124149 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] resource.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.124354 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.124525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.124714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.124899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.125141 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.125339 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.125529 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.125722 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.125906 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.126092 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.126275 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.126458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.126640 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.126841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.127028 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.127208 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.127390 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.127577 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.127761 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] role.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.127761 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] role.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.128116 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.44/identity {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.128325 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.128513 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] publish_errors = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.128712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.128899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.129085 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.129305 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.129488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.129678 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.129876 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.130059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.130241 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.130436 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.130621 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.130810 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] standard_threads = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.131005 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.131188 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.131371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.131564 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.131564 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.131896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.132092 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] use_journal = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.132277 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.132518 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] use_json = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.132518 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] use_stderr = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.132847 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.133030 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] use_syslog = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.133307 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.133477 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 19:47:08.133657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.133835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.133835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.134101 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.134242 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.134344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.134459 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.134574 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.134690 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.134815 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.134932 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.135047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.135163 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.135277 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.135393 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.135517 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.135634 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.135750 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.135866 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.135989 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.136112 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.136236 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.136369 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.136486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.136486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.136703 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.136831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.136958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.137076 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.137191 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.137305 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.137427 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.137544 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.137666 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.137783 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.137898 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.138013 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.138013 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.138229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.138356 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] token.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.138356 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.138573 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.138688 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.138803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.138919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.138919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.139145 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.139261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.139261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.139477 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.139591 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.139721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.139838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.139838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.140062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.140178 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.140347 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.140480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.140599 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.140714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.140830 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.140946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.141062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.141187 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.141304 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.141402 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.141499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.141623 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.141739 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.141739 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.141967 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.142084 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.142199 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.connection = **** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.142323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.142449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.142449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.142449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.142449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.142449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.142449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.142449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.142449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.142449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.145058 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.145177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.145295 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.145411 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.145527 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.145651 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.145770 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.145885 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.145885 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.146103 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.146103 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.146330 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.146330 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.146550 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.146666 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.146789 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.146908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.147032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.147151 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 23 19:47:08.147268 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.147268 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.147268 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.147268 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.147268 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.147268 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.147268 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.147268 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.148074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.148074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.148074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.148424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.148539 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.148751 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.149035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.149300 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.149548 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.149796 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.150039 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.150311 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.150557 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.153041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.153041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.153041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.153041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.153041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.153041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.153041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.153041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.153041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.153987 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.153987 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.153987 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.153987 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.153987 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.154525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.154525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.154525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.154840 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.154990 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.155255 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.155646 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.156062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.156530 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.156953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.157364 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.157782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.158174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.158574 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.158954 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.159330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.159710 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.160080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.160492 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.160878 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.161269 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.161648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.162051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.162473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.162853 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.163224 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] auth.external = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.163585 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.164054 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.164979 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.165379 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] auth.password = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.165748 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] auth.token = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.166132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.166506 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.166897 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.167267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.167658 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.168051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.168456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.168839 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] credential.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.169235 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.169628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.170017 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.170390 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.170773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.171160 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.171550 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.171955 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.172378 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.172782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.173223 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.173613 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.173879 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.174017 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.174136 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.174316 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.174439 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.174778 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.175154 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.175534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.175926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.176348 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.176710 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.177100 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.177471 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.177872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.178247 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.178641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.179013 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.179402 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.179773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.180790 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.181191 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.181574 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.181947 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.182338 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.182710 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.183089 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.183342 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.183465 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.183858 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.184211 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.184610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.184977 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.185363 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.185752 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.186140 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.186541 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.186933 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.187357 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.187991 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.188471 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.188789 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.189477 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.189477 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.190148 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.190148 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.190367 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.190631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.190999 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.191395 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.191815 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.192514 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.193055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.193055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.193273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.193583 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.193949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.194325 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.194726 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.195505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.195505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.196166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.196166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.196407 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.196769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.197133 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.197524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.197960 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.198304 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.198575 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.198889 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.199136 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.199516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.199919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.200278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.200715 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.201140 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.201522 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.201840 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.202077 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.202318 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.202556 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.202796 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.203028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.203276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.203526 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.203774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.204023 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.204271 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.204526 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.204776 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.205015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.205253 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.205489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.205736 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.205985 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.206236 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.206475 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.206732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.206968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.207214 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.207453 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.207691 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.207931 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.208180 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.208445 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.208683 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.208923 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.209166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.209418 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.209669 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.209908 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.210159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.210412 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.210652 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.210898 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.211138 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.211924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.212164 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] role.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.212441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] role.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.212617 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.212742 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.212858 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.212975 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.213158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.213409 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.213658 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.213906 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.214152 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.214415 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.214664 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.214899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.215148 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.215397 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.215646 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.215894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.216347 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.216592 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.216846 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.217095 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.217345 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.217595 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.217833 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.218089 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.218343 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.218584 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.218819 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.219054 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.219289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.219545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.219806 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.220059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.220314 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.220560 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.220796 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.221036 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.221276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.221527 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.221772 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.222031 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.222280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.222521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.222767 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.223011 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.223261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.223498 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.223740 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.223976 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.224223 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.224494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.224729 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.224973 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.225267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.225543 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.connection = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.226131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.226131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.226342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.226466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.226702 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.226942 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.227748 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.227990 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.228238 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.228498 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.228744 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.231035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.231035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.231035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.231035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.231035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.231035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.231035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 19:47:08.231035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 23 19:47:08.256188 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.256372 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.256612 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.265525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.284805 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.285142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.353899 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.404731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.404973 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.409092 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.444145 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.444405 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.469976 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.470211 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.470434 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.470653 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.470870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.471083 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.496690 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.496943 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.497091 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.497219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.497321 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.497419 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.497539 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.497665 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.497784 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.497900 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.498016 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.498132 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.498314 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.509406 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.519031 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.519374 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.519705 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.520028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.520376 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.520695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.521013 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.522189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.522416 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.522636 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.522852 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.523066 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.523277 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.523487 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.533359 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.537349 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.537685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.538009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.538362 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.538718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.538944 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.539197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 19:47:08.545449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.545660 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 19:47:08.545994 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.546193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 19:47:08.546460 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.546719 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.546974 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.547228 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.552207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.552444 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 19:47:08.552778 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.552978 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 19:47:08.553245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.553511 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.553766 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.554032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.558785 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.559255 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.559611 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.559906 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.566215 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.566506 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.566782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.567066 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.568805 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.572644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.572922 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.573205 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.573486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.573763 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.574019 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.574282 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.574551 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.574818 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.575089 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.575354 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.575795 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.579190 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.579476 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.579758 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.580142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.580452 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.580715 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.581000 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.581275 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.581565 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.593997 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.593997 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.602314 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.603926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.604206 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.609488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.609585 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.609873 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.610153 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.611110 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.611390 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.616729 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.617005 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.617281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.617471 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.617854 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.622362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.622646 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.622927 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.623207 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.623479 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.623752 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.624021 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.624285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.624628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.624913 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.629595 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.629879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.630163 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.630441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.630712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.630992 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.631259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.631532 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.643619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.643895 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.648557 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.651510 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.651785 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.652867 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.653261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.656355 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.658323 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.662069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.662069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.662216 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.665317 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.665683 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.665977 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.666261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.666544 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.666821 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.667096 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.669109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.672743 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.673036 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.673313 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.673587 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.673933 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.674213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.686082 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.686364 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.686629 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.686898 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.693329 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.693607 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.693875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.694132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.695086 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.697409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.697688 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.700946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.701231 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.701508 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.701783 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.702064 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.702450 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.704148 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.704446 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.707455 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.707753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.708144 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.708462 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.708736 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.715781 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.716059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.716355 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.716636 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.718186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.718476 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.718770 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.722677 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.722952 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.723232 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.724807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.724807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.724807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.724807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.736602 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.739891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.743232 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.743390 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.744999 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.746643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.746738 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.746924 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.747216 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.749958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.751543 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.753202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.753475 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.753756 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.754515 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.756111 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.759944 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.760184 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.760603 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.760883 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.761656 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.763300 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.766510 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.766794 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.767070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.767342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.769962 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.770246 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.770522 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.777098 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.777378 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.777660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.778949 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.779228 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.779505 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.779775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.780042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.780433 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.780758 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.781067 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.781335 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.781611 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.781883 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.782155 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.782416 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.782698 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.785499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 19:47:08.785778 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.786055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.786325 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.786591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.786864 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.787146 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.787409 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.787670 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.787950 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.788213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.788490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.788862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.789134 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 19:47:08.822498 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556657f1e9c0 pid: 80362 (default app) Jun 23 19:47:08.826082 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc259b31-633d-4344-8837-5a0672b081d0 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:08.826372 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc259b31-633d-4344-8837-5a0672b081d0 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:08.826646 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc259b31-633d-4344-8837-5a0672b081d0 None None] PATH_INFO: `/v3` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:08.827899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556657f1e9c0 pid: 80363 (default app) Jun 23 19:47:08.830866 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1/1] 10.222.0.44 () {50 vars in 791 bytes} [Thu Jun 23 19:47:08 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:18.022315 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2191878-dad1-4336-9603-6f45ec26f910 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:18.022866 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2191878-dad1-4336-9603-6f45ec26f910 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:18.022866 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2191878-dad1-4336-9603-6f45ec26f910 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:18.027825 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1/2] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:47:18.033226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e08256-7525-48e7-854c-88c2b40784d4 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:18.033582 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e08256-7525-48e7-854c-88c2b40784d4 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:18.033772 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e08256-7525-48e7-854c-88c2b40784d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:18.103462 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: INFO dbcounter [None req-95e08256-7525-48e7-854c-88c2b40784d4 None None] Registered counter for database keystone Jun 23 19:47:18.124281 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG oslo_db.sqlalchemy.engines [None req-95e08256-7525-48e7-854c-88c2b40784d4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80362) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 23 19:47:18.130598 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writer thread running {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 23 19:47:18.306691 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG passlib.handlers.bcrypt [None req-95e08256-7525-48e7-854c-88c2b40784d4 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80362) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 23 19:47:18.307052 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG passlib.handlers.bcrypt [None req-95e08256-7525-48e7-854c-88c2b40784d4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80362) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 23 19:47:18.362553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-95e08256-7525-48e7-854c-88c2b40784d4 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:18.364167 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-95e08256-7525-48e7-854c-88c2b40784d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:18.484966 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 2/3] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 453 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 23 19:47:18.609609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2191878-dad1-4336-9603-6f45ec26f910 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:18.678513 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: INFO dbcounter [None req-f2191878-dad1-4336-9603-6f45ec26f910 None None] Registered counter for database keystone Jun 23 19:47:18.698743 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG oslo_db.sqlalchemy.engines [None req-f2191878-dad1-4336-9603-6f45ec26f910 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80363) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 23 19:47:18.705189 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writer thread running {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 23 19:47:18.823124 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2191878-dad1-4336-9603-6f45ec26f910 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:18.827523 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac9920-5f30-45be-8dc3-b2a772cd9dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:18.828174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac9920-5f30-45be-8dc3-b2a772cd9dac admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:18.828489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac9920-5f30-45be-8dc3-b2a772cd9dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:18.828766 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac9920-5f30-45be-8dc3-b2a772cd9dac admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:18.836890 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbac9920-5f30-45be-8dc3-b2a772cd9dac admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:18.888257 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fbac9920-5f30-45be-8dc3-b2a772cd9dac admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 23 19:47:18.888564 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbac9920-5f30-45be-8dc3-b2a772cd9dac admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:18.897948 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 2/4] 10.222.0.44 () {58 vars in 1324 bytes} [Thu Jun 23 19:47:18 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 19:47:18.903995 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e08256-7525-48e7-854c-88c2b40784d4 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:18.918108 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e08256-7525-48e7-854c-88c2b40784d4 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:18.922483 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00cc439-7113-4397-8351-06d915519344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:18.923597 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00cc439-7113-4397-8351-06d915519344 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:18.923880 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00cc439-7113-4397-8351-06d915519344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:18.924152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00cc439-7113-4397-8351-06d915519344 admin admin] PATH_INFO: `/v3/projects/e3c8189cd37644e9a49e850123b33202` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:18.928174 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00cc439-7113-4397-8351-06d915519344 admin admin] RBAC: Authorizing `identity:get_project(project_id=e3c8189cd37644e9a49e850123b33202)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:18.982041 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d00cc439-7113-4397-8351-06d915519344 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 23 19:47:18.982375 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00cc439-7113-4397-8351-06d915519344 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:18.986934 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 3/5] 10.222.0.44 () {56 vars in 1207 bytes} [Thu Jun 23 19:47:18 2022] GET /identity/v3/projects/e3c8189cd37644e9a49e850123b33202 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:47:20.950941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c878547-dde7-40c3-8722-77dcbb6a6c8b None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:20.951454 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c878547-dde7-40c3-8722-77dcbb6a6c8b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:20.951565 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c878547-dde7-40c3-8722-77dcbb6a6c8b None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:20.955620 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 3/6] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:20.960805 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec79f2-55ae-473e-9a04-f949383ae0aa None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:20.961694 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec79f2-55ae-473e-9a04-f949383ae0aa None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:20.961901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec79f2-55ae-473e-9a04-f949383ae0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:20.980771 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d6ec79f2-55ae-473e-9a04-f949383ae0aa None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:20.982286 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d6ec79f2-55ae-473e-9a04-f949383ae0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:20.990013 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 4/7] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:20 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 23 19:47:21.012003 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c878547-dde7-40c3-8722-77dcbb6a6c8b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:21.024258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c878547-dde7-40c3-8722-77dcbb6a6c8b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:21.028543 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150eba0c-28c4-4f8c-81c5-9b112ff24164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:21.029154 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150eba0c-28c4-4f8c-81c5-9b112ff24164 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:21.029433 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150eba0c-28c4-4f8c-81c5-9b112ff24164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:21.029703 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150eba0c-28c4-4f8c-81c5-9b112ff24164 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:21.037257 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150eba0c-28c4-4f8c-81c5-9b112ff24164 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:21.039026 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150eba0c-28c4-4f8c-81c5-9b112ff24164 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:21.048164 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 4/8] 10.222.0.44 () {58 vars in 1324 bytes} [Thu Jun 23 19:47:21 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 19:47:21.053325 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec79f2-55ae-473e-9a04-f949383ae0aa None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:21.062185 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec79f2-55ae-473e-9a04-f949383ae0aa None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:21.066858 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8b508-1b94-4ad8-ab28-a355047e6b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:21.067712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8b508-1b94-4ad8-ab28-a355047e6b7a admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:21.067999 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8b508-1b94-4ad8-ab28-a355047e6b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:21.068279 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8b508-1b94-4ad8-ab28-a355047e6b7a admin admin] PATH_INFO: `/v3/users/5060168fc25744178d345904afb6c333` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:21.069701 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a8b508-1b94-4ad8-ab28-a355047e6b7a admin admin] RBAC: Authorizing `identity:get_user(user_id=5060168fc25744178d345904afb6c333)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:21.071527 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a8b508-1b94-4ad8-ab28-a355047e6b7a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:21.073111 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 5/9] 10.222.0.44 () {56 vars in 1198 bytes} [Thu Jun 23 19:47:21 2022] GET /identity/v3/users/5060168fc25744178d345904afb6c333 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:23.142548 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abd616-cd74-4347-92e4-42bef52c459a None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:23.143061 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abd616-cd74-4347-92e4-42bef52c459a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:23.143142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abd616-cd74-4347-92e4-42bef52c459a None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:23.144653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 5/10] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:23.150206 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33b8cc-02ac-4ef2-bcec-fc2b84f1386f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:23.150612 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33b8cc-02ac-4ef2-bcec-fc2b84f1386f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:23.150899 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33b8cc-02ac-4ef2-bcec-fc2b84f1386f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:23.167510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ac33b8cc-02ac-4ef2-bcec-fc2b84f1386f None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:23.168739 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ac33b8cc-02ac-4ef2-bcec-fc2b84f1386f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:23.180431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520567e3-d304-4f2a-89c0-e977090ad8f4 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:23.180731 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520567e3-d304-4f2a-89c0-e977090ad8f4 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:23.181011 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520567e3-d304-4f2a-89c0-e977090ad8f4 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:23.182402 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 6/11] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:47:23.187348 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcac4a9-289d-4812-8094-201ad0c39d53 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:23.187638 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcac4a9-289d-4812-8094-201ad0c39d53 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:23.188177 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcac4a9-289d-4812-8094-201ad0c39d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:23.227287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG passlib.handlers.bcrypt [None req-ffcac4a9-289d-4812-8094-201ad0c39d53 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80363) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 23 19:47:23.227724 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG passlib.handlers.bcrypt [None req-ffcac4a9-289d-4812-8094-201ad0c39d53 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80363) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 23 19:47:23.250188 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 6/12] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:23.274678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33b8cc-02ac-4ef2-bcec-fc2b84f1386f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:23.282831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ffcac4a9-289d-4812-8094-201ad0c39d53 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:23.285279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ffcac4a9-289d-4812-8094-201ad0c39d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:23.299370 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:23.379145 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33b8cc-02ac-4ef2-bcec-fc2b84f1386f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:23.393562 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 7/13] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 19:47:23.412063 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c6db61-141f-4fc6-b831-db2d9c7e4b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:23.412977 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6db61-141f-4fc6-b831-db2d9c7e4b58 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:23.413331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6db61-141f-4fc6-b831-db2d9c7e4b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:23.413770 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6db61-141f-4fc6-b831-db2d9c7e4b58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:23.414779 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcac4a9-289d-4812-8094-201ad0c39d53 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:23.425603 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c6db61-141f-4fc6-b831-db2d9c7e4b58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:47:23.437254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-e1c6db61-141f-4fc6-b831-db2d9c7e4b58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:47:23.437254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:47:23.437254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:47:23.437254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:47:23.437254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:47:23.437254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:47:23.437254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:47:23.437254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:47:23.437254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:47:23.437254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:47:23.437254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:47:23.437254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:47:23.437254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:47:23.437254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:47:23.437254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:47:23.437254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:47:23.438526 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:47:23.438526 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:47:23.438526 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:47:23.438526 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:47:23.438526 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:47:23.438526 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:47:23.438526 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:47:23.438526 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:47:23.438526 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:47:23.438526 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:47:23.438526 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:47:23.438526 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:47:23.438526 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:47:23.438526 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:47:23.438526 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:47:23.438526 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:47:23.439363 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:47:23.439363 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:47:23.439363 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:47:23.439363 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:47:23.439363 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:47:23.439363 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:47:23.439363 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:47:23.439363 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:47:23.439363 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:47:23.439363 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:47:23.439363 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:47:23.439363 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:47:23.439363 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 7/14] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:47:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:47:23.444470 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c6db61-141f-4fc6-b831-db2d9c7e4b58 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:23.517836 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcac4a9-289d-4812-8094-201ad0c39d53 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:23.520090 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:23.523696 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:23.541390 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c6db61-141f-4fc6-b831-db2d9c7e4b58 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:23.554467 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65af3c6-7fe9-43b7-b25d-6ce164c9e1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:23.555920 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65af3c6-7fe9-43b7-b25d-6ce164c9e1ed None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:23.556202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65af3c6-7fe9-43b7-b25d-6ce164c9e1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:23.556691 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65af3c6-7fe9-43b7-b25d-6ce164c9e1ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:23.575586 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede45bd4-a863-402f-90f6-711da996bf77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:23.576396 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede45bd4-a863-402f-90f6-711da996bf77 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:23.576710 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede45bd4-a863-402f-90f6-711da996bf77 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:23.576986 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede45bd4-a863-402f-90f6-711da996bf77 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:23.577735 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede45bd4-a863-402f-90f6-711da996bf77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:23.578039 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede45bd4-a863-402f-90f6-711da996bf77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:23.579446 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede45bd4-a863-402f-90f6-711da996bf77 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:23.590998 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 8/15] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:47:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:47:23.596388 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede45bd4-a863-402f-90f6-711da996bf77 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:23.627570 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65af3c6-7fe9-43b7-b25d-6ce164c9e1ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:23.630942 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65af3c6-7fe9-43b7-b25d-6ce164c9e1ed None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:23.698872 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede45bd4-a863-402f-90f6-711da996bf77 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:23.731447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175dd59-367c-4421-b58f-48d00c682321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:23.732328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175dd59-367c-4421-b58f-48d00c682321 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:23.732620 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175dd59-367c-4421-b58f-48d00c682321 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:23.732860 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 8/16] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:47:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 19:47:23.732951 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9175dd59-367c-4421-b58f-48d00c682321 None admin] PATH_INFO: `/v3/users/5060168fc25744178d345904afb6c333` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:23.734093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9175dd59-367c-4421-b58f-48d00c682321 None admin] RBAC: Authorizing `identity:get_user(user_id=5060168fc25744178d345904afb6c333)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:23.735566 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9175dd59-367c-4421-b58f-48d00c682321 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:23.740334 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 9/17] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:47:23 2022] GET /identity/v3/users/5060168fc25744178d345904afb6c333 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:47:23.740483 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65af3c6-7fe9-43b7-b25d-6ce164c9e1ed None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:23.741824 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:23.743621 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175dd59-367c-4421-b58f-48d00c682321 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:23.757635 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:23.835867 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65af3c6-7fe9-43b7-b25d-6ce164c9e1ed None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:23.840912 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9175dd59-367c-4421-b58f-48d00c682321 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:23.868949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d40f31-17a8-41b8-b368-a7a76723afb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:23.869742 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d40f31-17a8-41b8-b368-a7a76723afb9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:23.870029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d40f31-17a8-41b8-b368-a7a76723afb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:23.870327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d40f31-17a8-41b8-b368-a7a76723afb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:23.873838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cceea1-036f-42ee-9375-2620fe6806fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:23.874608 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cceea1-036f-42ee-9375-2620fe6806fd None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:23.874889 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cceea1-036f-42ee-9375-2620fe6806fd None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:23.875160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cceea1-036f-42ee-9375-2620fe6806fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:23.876371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cceea1-036f-42ee-9375-2620fe6806fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:23.877867 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cceea1-036f-42ee-9375-2620fe6806fd None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:23.879462 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 10/18] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:47:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:47:23.885228 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cceea1-036f-42ee-9375-2620fe6806fd None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:23.889791 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d40f31-17a8-41b8-b368-a7a76723afb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:23.891263 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d40f31-17a8-41b8-b368-a7a76723afb9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:23.912043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-09d40f31-17a8-41b8-b368-a7a76723afb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:47:23.912043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:47:23.912043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:47:23.912043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:47:23.912043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:47:23.912043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:47:23.912043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:47:23.912043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:47:23.912043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:47:23.912043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:47:23.912043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:47:23.912043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:47:23.912043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:47:23.912043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:47:23.912043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:47:23.912903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:47:23.912903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:47:23.912903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:47:23.912903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:47:23.912903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:47:23.912903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:47:23.912903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:47:23.912903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:47:23.912903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:47:23.912903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:47:23.912903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:47:23.912903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:47:23.912903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:47:23.912903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:47:23.912903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:47:23.912903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:47:23.912903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:47:23.913697 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:47:23.913697 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:47:23.913697 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:47:23.913697 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:47:23.913697 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 9/19] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:47:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:47:23.919480 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d40f31-17a8-41b8-b368-a7a76723afb9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:23.967683 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:23.977993 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:23.990138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cceea1-036f-42ee-9375-2620fe6806fd None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:24.019435 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d40f31-17a8-41b8-b368-a7a76723afb9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:24.022570 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41e63e-4b9c-4a06-8177-b28e6a5d93a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:24.023222 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41e63e-4b9c-4a06-8177-b28e6a5d93a7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:24.024143 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41e63e-4b9c-4a06-8177-b28e6a5d93a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:24.024462 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41e63e-4b9c-4a06-8177-b28e6a5d93a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:24.025299 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d41e63e-4b9c-4a06-8177-b28e6a5d93a7 None admin] RBAC: Adding query filter params (role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb, scope.domain.id=default, user.id=5060168fc25744178d345904afb6c333) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:24.025609 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d41e63e-4b9c-4a06-8177-b28e6a5d93a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:24.027063 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d41e63e-4b9c-4a06-8177-b28e6a5d93a7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:24.033639 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 11/20] 10.222.0.44 () {56 vars in 1322 bytes} [Thu Jun 23 19:47:23 2022] GET /identity/v3/role_assignments?user.id=5060168fc25744178d345904afb6c333&scope.domain.id=default&role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:47:24.053824 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99831c81-607b-42ef-8ff7-e48cca7a0285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:24.054507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99831c81-607b-42ef-8ff7-e48cca7a0285 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:24.054820 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99831c81-607b-42ef-8ff7-e48cca7a0285 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:24.055093 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99831c81-607b-42ef-8ff7-e48cca7a0285 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:24.055849 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99831c81-607b-42ef-8ff7-e48cca7a0285 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:24.056155 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99831c81-607b-42ef-8ff7-e48cca7a0285 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:24.057775 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99831c81-607b-42ef-8ff7-e48cca7a0285 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:24.078374 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 10/21] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:47:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:47:26.453889 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff02b88-0965-49b8-8cb8-0d826537b469 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:26.454420 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff02b88-0965-49b8-8cb8-0d826537b469 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:26.454496 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff02b88-0965-49b8-8cb8-0d826537b469 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:26.456106 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 12/22] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:26.461859 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa8eda-9b4c-4c3d-92b8-57604388c9e7 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:26.462702 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa8eda-9b4c-4c3d-92b8-57604388c9e7 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:26.463201 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa8eda-9b4c-4c3d-92b8-57604388c9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:26.489700 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e3fa8eda-9b4c-4c3d-92b8-57604388c9e7 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:26.491214 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e3fa8eda-9b4c-4c3d-92b8-57604388c9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:26.496743 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f8557-272f-4af7-8333-af04315f3532 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:26.497300 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f8557-272f-4af7-8333-af04315f3532 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:26.497803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f8557-272f-4af7-8333-af04315f3532 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:26.499719 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 13/23] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:26.509497 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1be392-dd5d-403d-a586-e18f23ba67ec None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:26.510067 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1be392-dd5d-403d-a586-e18f23ba67ec None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:26.510586 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1be392-dd5d-403d-a586-e18f23ba67ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:26.545288 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-5a1be392-dd5d-403d-a586-e18f23ba67ec None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:26.546830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5a1be392-dd5d-403d-a586-e18f23ba67ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:26.607929 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 11/24] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:26.644415 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa8eda-9b4c-4c3d-92b8-57604388c9e7 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:26.678829 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 14/25] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:26.727737 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b19db-5e27-45fd-9526-55c9ac1c3846 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:26.728394 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b19db-5e27-45fd-9526-55c9ac1c3846 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:26.728867 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b19db-5e27-45fd-9526-55c9ac1c3846 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:26.730430 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 15/26] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:47:26.733557 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:26.737013 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b19db-5e27-45fd-9526-55c9ac1c3846 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:26.794642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa8eda-9b4c-4c3d-92b8-57604388c9e7 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:26.837080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96be5c8-b926-42b5-97ea-0cd763b3e16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:26.838120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96be5c8-b926-42b5-97ea-0cd763b3e16b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:26.843716 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96be5c8-b926-42b5-97ea-0cd763b3e16b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:26.844171 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96be5c8-b926-42b5-97ea-0cd763b3e16b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:26.861437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96be5c8-b926-42b5-97ea-0cd763b3e16b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:47:26.875795 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-d96be5c8-b926-42b5-97ea-0cd763b3e16b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:47:26.875795 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:47:26.875795 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:47:26.875795 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:47:26.875795 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:47:26.875795 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:47:26.875795 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:47:26.875795 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:47:26.875795 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:47:26.875795 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:47:26.875795 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:47:26.875795 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:47:26.875795 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:47:26.875795 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:47:26.875795 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:47:26.875795 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:47:26.876899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:47:26.876899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:47:26.876899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:47:26.876899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:47:26.876899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:47:26.876899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:47:26.876899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:47:26.876899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:47:26.876899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:47:26.876899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:47:26.876899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:47:26.876899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:47:26.876899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:47:26.876899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:47:26.876899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:47:26.876899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:47:26.877847 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:47:26.877847 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:47:26.877847 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:47:26.877847 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:47:26.877847 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:47:26.877847 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:47:26.877847 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:47:26.877847 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:47:26.877847 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:47:26.877847 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:47:26.877847 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:47:26.877847 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:47:26.880331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 12/27] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:47:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:47:26.881829 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77fdee-364a-4d10-b379-6f29c62d73cc None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:26.882263 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77fdee-364a-4d10-b379-6f29c62d73cc None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:26.882717 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77fdee-364a-4d10-b379-6f29c62d73cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:26.903509 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5c77fdee-364a-4d10-b379-6f29c62d73cc None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:26.905022 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5c77fdee-364a-4d10-b379-6f29c62d73cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:26.924678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:27.012204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 13/28] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:27.013332 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77fdee-364a-4d10-b379-6f29c62d73cc None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:27.013644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b19db-5e27-45fd-9526-55c9ac1c3846 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:27.051093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd7408-3d8f-48ff-9afa-3163c4e1e3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:27.051951 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd7408-3d8f-48ff-9afa-3163c4e1e3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:27.052347 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd7408-3d8f-48ff-9afa-3163c4e1e3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:27.052722 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd7408-3d8f-48ff-9afa-3163c4e1e3f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:27.064435 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd7408-3d8f-48ff-9afa-3163c4e1e3f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:27.066055 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd7408-3d8f-48ff-9afa-3163c4e1e3f4 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:27.067239 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:27.085657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-debd7408-3d8f-48ff-9afa-3163c4e1e3f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:47:27.085657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:47:27.085657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:47:27.085657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:47:27.085657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:47:27.085657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:47:27.085657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:47:27.085657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:47:27.085657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:47:27.085657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:47:27.085657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:47:27.085657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:47:27.085657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:47:27.085657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:47:27.085657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:47:27.087232 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:47:27.087232 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:47:27.087232 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:47:27.087232 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:47:27.087232 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:47:27.087232 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:47:27.087232 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:47:27.087232 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:47:27.087232 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:47:27.087232 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:47:27.087232 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:47:27.087232 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:47:27.087232 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:47:27.087232 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:47:27.087232 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:47:27.087232 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:47:27.087232 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:47:27.088243 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:47:27.088243 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:47:27.088243 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:47:27.088243 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:47:27.089541 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 16/29] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:47:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:47:27.091006 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84851e-809a-4787-abd5-16dcb4903bee None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:27.091382 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84851e-809a-4787-abd5-16dcb4903bee None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:27.091874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84851e-809a-4787-abd5-16dcb4903bee None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:27.093265 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 17/30] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:27.098278 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84851e-809a-4787-abd5-16dcb4903bee None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:27.147928 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77fdee-364a-4d10-b379-6f29c62d73cc None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:27.190661 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c03178-823f-4ddb-ac7f-059deed1151d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:27.191806 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c03178-823f-4ddb-ac7f-059deed1151d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:27.192236 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c03178-823f-4ddb-ac7f-059deed1151d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:27.192678 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c03178-823f-4ddb-ac7f-059deed1151d None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:27.193739 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c03178-823f-4ddb-ac7f-059deed1151d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:27.194488 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c03178-823f-4ddb-ac7f-059deed1151d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:27.196067 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c03178-823f-4ddb-ac7f-059deed1151d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:27.210507 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84851e-809a-4787-abd5-16dcb4903bee None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:27.211990 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 14/31] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:47:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:27.213110 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c03178-823f-4ddb-ac7f-059deed1151d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:27.243373 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:27.252967 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0efbad-6b7b-4b52-ba21-66a2846c9fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:27.253899 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0efbad-6b7b-4b52-ba21-66a2846c9fbc None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:27.254375 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0efbad-6b7b-4b52-ba21-66a2846c9fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:27.254866 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0efbad-6b7b-4b52-ba21-66a2846c9fbc None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:27.255801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0efbad-6b7b-4b52-ba21-66a2846c9fbc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:27.257541 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0efbad-6b7b-4b52-ba21-66a2846c9fbc None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:27.345291 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:27.349608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c03178-823f-4ddb-ac7f-059deed1151d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:27.405664 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b154e503-df0b-455c-8bc1-b8480b7ff6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:27.406840 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154e503-df0b-455c-8bc1-b8480b7ff6d1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:27.407138 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154e503-df0b-455c-8bc1-b8480b7ff6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:27.407508 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154e503-df0b-455c-8bc1-b8480b7ff6d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:27.408825 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b154e503-df0b-455c-8bc1-b8480b7ff6d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:27.409388 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b154e503-df0b-455c-8bc1-b8480b7ff6d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:27.411651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b154e503-df0b-455c-8bc1-b8480b7ff6d1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:27.425644 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 15/32] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:47:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:27.427172 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a10707-7a1c-4d0a-9838-c1cae8ffdba2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:27.427470 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a10707-7a1c-4d0a-9838-c1cae8ffdba2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:27.427800 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a10707-7a1c-4d0a-9838-c1cae8ffdba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:27.446723 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-00a10707-7a1c-4d0a-9838-c1cae8ffdba2 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:27.447983 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-00a10707-7a1c-4d0a-9838-c1cae8ffdba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:27.487457 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 18/33] 10.222.0.44 () {60 vars in 1131 bytes} [Thu Jun 23 19:47:27 2022] POST /identity/v3/roles => generated 219 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:47:27.489231 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0efbad-6b7b-4b52-ba21-66a2846c9fbc None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:27.534206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 16/34] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:27.535794 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a10707-7a1c-4d0a-9838-c1cae8ffdba2 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:27.647148 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:27.662250 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a10707-7a1c-4d0a-9838-c1cae8ffdba2 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:27.704955 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0efbad-6b7b-4b52-ba21-66a2846c9fbc None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:27.708707 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6d332-8f1a-44e4-80ac-fa0cd107a3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:27.709683 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6d332-8f1a-44e4-80ac-fa0cd107a3d0 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:27.710168 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6d332-8f1a-44e4-80ac-fa0cd107a3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:27.710660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6d332-8f1a-44e4-80ac-fa0cd107a3d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:27.711833 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a6d332-8f1a-44e4-80ac-fa0cd107a3d0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:27.713767 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a6d332-8f1a-44e4-80ac-fa0cd107a3d0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:27.743871 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77c750-9913-4dde-b815-039c437e4be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:27.744770 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77c750-9913-4dde-b815-039c437e4be9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:27.745308 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77c750-9913-4dde-b815-039c437e4be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:27.745652 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77c750-9913-4dde-b815-039c437e4be9 None admin] PATH_INFO: `/v3/users/5060168fc25744178d345904afb6c333` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:27.747001 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b77c750-9913-4dde-b815-039c437e4be9 None admin] RBAC: Authorizing `identity:get_user(user_id=5060168fc25744178d345904afb6c333)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:27.749228 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b77c750-9913-4dde-b815-039c437e4be9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:27.750613 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 19/35] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:47:27 2022] GET /identity/v3/users/5060168fc25744178d345904afb6c333 => generated 247 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:27.751256 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77c750-9913-4dde-b815-039c437e4be9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:27.817343 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:27.861279 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77c750-9913-4dde-b815-039c437e4be9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:27.898257 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b4bd0-1281-4e27-a280-dbae4d95d9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:27.900269 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b4bd0-1281-4e27-a280-dbae4d95d9d7 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:27.900791 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b4bd0-1281-4e27-a280-dbae4d95d9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:27.901067 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b4bd0-1281-4e27-a280-dbae4d95d9d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:27.901821 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b4bd0-1281-4e27-a280-dbae4d95d9d7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:27.903271 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b4bd0-1281-4e27-a280-dbae4d95d9d7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:27.982727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 17/36] 10.222.0.44 () {60 vars in 1141 bytes} [Thu Jun 23 19:47:27 2022] POST /identity/v3/projects => generated 301 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:47:27.984023 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6d332-8f1a-44e4-80ac-fa0cd107a3d0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:27.997479 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 20/37] 10.222.0.44 () {60 vars in 1131 bytes} [Thu Jun 23 19:47:27 2022] POST /identity/v3/roles => generated 225 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:47:28.006975 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817192b2-5529-4c3e-9084-a5245900cc3e None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:28.007558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817192b2-5529-4c3e-9084-a5245900cc3e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:28.008775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817192b2-5529-4c3e-9084-a5245900cc3e None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:28.013901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 21/38] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:28.019693 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640313ca-5fe6-4901-95af-490ccf0c2c5c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:28.020324 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640313ca-5fe6-4901-95af-490ccf0c2c5c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:28.020897 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640313ca-5fe6-4901-95af-490ccf0c2c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:28.049761 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-640313ca-5fe6-4901-95af-490ccf0c2c5c None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:28.051837 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-640313ca-5fe6-4901-95af-490ccf0c2c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:28.108050 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6d332-8f1a-44e4-80ac-fa0cd107a3d0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:28.155430 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 22/39] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:28.156965 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6e0c6-fb12-482b-be2b-b55575476613 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:28.157384 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6e0c6-fb12-482b-be2b-b55575476613 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:28.157670 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6e0c6-fb12-482b-be2b-b55575476613 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:28.160382 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 23/40] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:28.165963 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83eb27c-c839-49e4-8e75-444dd58046ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:28.166160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ca98d-7a62-4e90-a94b-24a7665b920f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:28.166322 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ca98d-7a62-4e90-a94b-24a7665b920f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:28.166582 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ca98d-7a62-4e90-a94b-24a7665b920f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:28.167474 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83eb27c-c839-49e4-8e75-444dd58046ba None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:28.167960 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83eb27c-c839-49e4-8e75-444dd58046ba None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:28.168499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83eb27c-c839-49e4-8e75-444dd58046ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:28.170086 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83eb27c-c839-49e4-8e75-444dd58046ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:28.172414 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83eb27c-c839-49e4-8e75-444dd58046ba None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:28.175468 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 18/41] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:47:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:28.183945 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83eb27c-c839-49e4-8e75-444dd58046ba None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:28.190656 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-386ca98d-7a62-4e90-a94b-24a7665b920f None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:28.192453 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-386ca98d-7a62-4e90-a94b-24a7665b920f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:28.200782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:28.244647 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:28.275278 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 24/42] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 19:47:28.276085 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ca98d-7a62-4e90-a94b-24a7665b920f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:28.311601 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83eb27c-c839-49e4-8e75-444dd58046ba None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:28.383120 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ca98d-7a62-4e90-a94b-24a7665b920f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:28.420073 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91efc64-722f-4390-8215-da569bf465c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:28.421015 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91efc64-722f-4390-8215-da569bf465c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:28.421298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91efc64-722f-4390-8215-da569bf465c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:28.421577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91efc64-722f-4390-8215-da569bf465c1 None admin] PATH_INFO: `/v3/domains/default/users/5060168fc25744178d345904afb6c333/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:28.423705 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91efc64-722f-4390-8215-da569bf465c1 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=5060168fc25744178d345904afb6c333, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:28.425220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91efc64-722f-4390-8215-da569bf465c1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:28.450941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d70747-b4c4-48cb-acb3-015391d79436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:28.451605 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d70747-b4c4-48cb-acb3-015391d79436 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:28.451885 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d70747-b4c4-48cb-acb3-015391d79436 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:28.452166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d70747-b4c4-48cb-acb3-015391d79436 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:28.452940 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d70747-b4c4-48cb-acb3-015391d79436 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:28.454306 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d70747-b4c4-48cb-acb3-015391d79436 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:28.565507 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 25/43] 10.222.0.44 () {58 vars in 1361 bytes} [Thu Jun 23 19:47:28 2022] PUT /identity/v3/domains/default/users/5060168fc25744178d345904afb6c333/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:47:28.566340 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91efc64-722f-4390-8215-da569bf465c1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:28.597861 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 19/44] 10.222.0.44 () {60 vars in 1131 bytes} [Thu Jun 23 19:47:28 2022] POST /identity/v3/roles => generated 223 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:47:28.609237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:28.792407 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91efc64-722f-4390-8215-da569bf465c1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:28.828067 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bda4f9-058d-4ce6-966d-81f8f87d6722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:28.828902 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bda4f9-058d-4ce6-966d-81f8f87d6722 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:28.829190 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bda4f9-058d-4ce6-966d-81f8f87d6722 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:28.829469 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bda4f9-058d-4ce6-966d-81f8f87d6722 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:28.830655 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bda4f9-058d-4ce6-966d-81f8f87d6722 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:28.832158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bda4f9-058d-4ce6-966d-81f8f87d6722 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:28.834000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 26/45] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:47:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:28.840244 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d70747-b4c4-48cb-acb3-015391d79436 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:28.907101 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:28.939277 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d70747-b4c4-48cb-acb3-015391d79436 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:28.973418 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3be7c-4eb0-493b-a06e-8796a4dd257f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:28.974082 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3be7c-4eb0-493b-a06e-8796a4dd257f None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:28.974362 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3be7c-4eb0-493b-a06e-8796a4dd257f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:28.974636 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3be7c-4eb0-493b-a06e-8796a4dd257f None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:28.975344 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3be7c-4eb0-493b-a06e-8796a4dd257f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:28.976845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3be7c-4eb0-493b-a06e-8796a4dd257f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:29.093923 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 20/46] 10.222.0.44 () {60 vars in 1141 bytes} [Thu Jun 23 19:47:28 2022] POST /identity/v3/projects => generated 312 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 19:47:30.613174 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e572e1-8ee6-4050-9974-213e327ec90c None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:30.613682 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e572e1-8ee6-4050-9974-213e327ec90c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:30.613788 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e572e1-8ee6-4050-9974-213e327ec90c None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:30.615090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 27/47] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:30.620388 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250fd7f-711a-4ebb-9918-38da2859151c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:30.620767 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250fd7f-711a-4ebb-9918-38da2859151c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:30.621077 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250fd7f-711a-4ebb-9918-38da2859151c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:30.638399 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f250fd7f-711a-4ebb-9918-38da2859151c None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:30.639924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f250fd7f-711a-4ebb-9918-38da2859151c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:30.705718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 21/48] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:30.728958 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e572e1-8ee6-4050-9974-213e327ec90c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:30.831166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e572e1-8ee6-4050-9974-213e327ec90c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:30.832738 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:30.863779 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011522d-9a81-4b84-b8d3-4cbf09b25f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:30.864529 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011522d-9a81-4b84-b8d3-4cbf09b25f84 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:30.864818 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011522d-9a81-4b84-b8d3-4cbf09b25f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:30.865090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011522d-9a81-4b84-b8d3-4cbf09b25f84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:30.875714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9011522d-9a81-4b84-b8d3-4cbf09b25f84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:47:30.883851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-9011522d-9a81-4b84-b8d3-4cbf09b25f84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:47:30.883851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:47:30.883851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:47:30.883851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:47:30.883851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:47:30.883851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:47:30.883851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:47:30.883851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:47:30.883851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:47:30.883851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:47:30.883851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:47:30.883851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:47:30.883851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:47:30.883851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:47:30.883851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:47:30.883851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:47:30.884747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:47:30.884747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:47:30.884747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:47:30.884747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:47:30.884747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:47:30.884747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:47:30.884747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:47:30.884747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:47:30.884747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:47:30.884747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:47:30.884747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:47:30.884747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:47:30.884747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:47:30.884747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:47:30.884747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:47:30.884747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:47:30.885496 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:47:30.885496 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:47:30.885496 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:47:30.885496 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:47:30.885496 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:47:30.885496 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:47:30.885496 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:47:30.885496 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:47:30.885496 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:47:30.885496 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:47:30.885496 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:47:30.885496 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:47:30.885496 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 28/49] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:47:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:47:30.890804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250fd7f-711a-4ebb-9918-38da2859151c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:30.939941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:30.988838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250fd7f-711a-4ebb-9918-38da2859151c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:31.022558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f10e23-6ad1-4807-993a-03cda4623466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:31.023219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f10e23-6ad1-4807-993a-03cda4623466 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:31.023504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f10e23-6ad1-4807-993a-03cda4623466 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:31.023779 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f10e23-6ad1-4807-993a-03cda4623466 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:31.024552 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f10e23-6ad1-4807-993a-03cda4623466 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:31.024859 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f10e23-6ad1-4807-993a-03cda4623466 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:31.026253 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f10e23-6ad1-4807-993a-03cda4623466 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:31.034186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 22/50] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:47:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:31.040219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011522d-9a81-4b84-b8d3-4cbf09b25f84 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:31.138466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011522d-9a81-4b84-b8d3-4cbf09b25f84 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:31.152327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9fe99-5a7d-47bd-b415-14a14e813165 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:31.152688 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9fe99-5a7d-47bd-b415-14a14e813165 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:31.153058 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9fe99-5a7d-47bd-b415-14a14e813165 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:31.154518 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 23/51] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:47:31.159229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab5da8-62e3-4be6-a0e3-cbe95342e320 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:31.159516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab5da8-62e3-4be6-a0e3-cbe95342e320 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:31.159797 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab5da8-62e3-4be6-a0e3-cbe95342e320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:31.171674 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bcdf34-9764-4ead-9eea-ce96643ad3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:31.172589 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bcdf34-9764-4ead-9eea-ce96643ad3b1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:31.172878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bcdf34-9764-4ead-9eea-ce96643ad3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:31.173156 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bcdf34-9764-4ead-9eea-ce96643ad3b1 None admin] PATH_INFO: `/v3/users/5060168fc25744178d345904afb6c333` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:31.174354 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bcdf34-9764-4ead-9eea-ce96643ad3b1 None admin] RBAC: Authorizing `identity:get_user(user_id=5060168fc25744178d345904afb6c333)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:31.175820 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bcdf34-9764-4ead-9eea-ce96643ad3b1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:31.175905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1eab5da8-62e3-4be6-a0e3-cbe95342e320 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:31.177035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1eab5da8-62e3-4be6-a0e3-cbe95342e320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:31.177504 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 29/52] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:47:31 2022] GET /identity/v3/users/5060168fc25744178d345904afb6c333 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:31.182593 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bcdf34-9764-4ead-9eea-ce96643ad3b1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:31.213160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:31.244124 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 24/53] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 19:47:31.267558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab5da8-62e3-4be6-a0e3-cbe95342e320 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:31.283732 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bcdf34-9764-4ead-9eea-ce96643ad3b1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:31.310134 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:31.317741 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ede19a-65bb-4dee-b7e5-7e1537216eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:31.318530 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ede19a-65bb-4dee-b7e5-7e1537216eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:31.319649 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ede19a-65bb-4dee-b7e5-7e1537216eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:31.319649 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ede19a-65bb-4dee-b7e5-7e1537216eb3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:31.321013 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ede19a-65bb-4dee-b7e5-7e1537216eb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:31.322487 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ede19a-65bb-4dee-b7e5-7e1537216eb3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:31.324107 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 30/54] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:47:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:47:31.329840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ede19a-65bb-4dee-b7e5-7e1537216eb3 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:31.367511 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab5da8-62e3-4be6-a0e3-cbe95342e320 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:31.401645 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d40b6cc-de51-412e-b843-76c46185073b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:31.402397 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40b6cc-de51-412e-b843-76c46185073b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:31.402678 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40b6cc-de51-412e-b843-76c46185073b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:31.402951 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40b6cc-de51-412e-b843-76c46185073b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:31.404134 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d40b6cc-de51-412e-b843-76c46185073b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:31.405614 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d40b6cc-de51-412e-b843-76c46185073b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:31.407412 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 25/55] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:47:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:47:31.412698 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d40b6cc-de51-412e-b843-76c46185073b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:31.434570 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:31.438592 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ede19a-65bb-4dee-b7e5-7e1537216eb3 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:31.472414 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f1ef1-32e8-4c63-9767-e3ab6c495aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:31.473288 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f1ef1-32e8-4c63-9767-e3ab6c495aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:31.473577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f1ef1-32e8-4c63-9767-e3ab6c495aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:31.473854 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f1ef1-32e8-4c63-9767-e3ab6c495aa7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:31.474693 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44f1ef1-32e8-4c63-9767-e3ab6c495aa7 None admin] RBAC: Adding query filter params (role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb, scope.domain.id=default, user.id=5060168fc25744178d345904afb6c333) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:31.475007 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44f1ef1-32e8-4c63-9767-e3ab6c495aa7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:31.476526 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44f1ef1-32e8-4c63-9767-e3ab6c495aa7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:31.482226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 31/56] 10.222.0.44 () {56 vars in 1322 bytes} [Thu Jun 23 19:47:31 2022] GET /identity/v3/role_assignments?user.id=5060168fc25744178d345904afb6c333&scope.domain.id=default&role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 530 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:47:31.514509 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d40b6cc-de51-412e-b843-76c46185073b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:31.526504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:31.550780 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae833810-9366-4c93-a719-03706b85127f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:31.551456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae833810-9366-4c93-a719-03706b85127f None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:31.551742 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae833810-9366-4c93-a719-03706b85127f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:31.552069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae833810-9366-4c93-a719-03706b85127f None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:31.552880 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae833810-9366-4c93-a719-03706b85127f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:31.554906 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae833810-9366-4c93-a719-03706b85127f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:31.794285 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 26/57] 10.222.0.44 () {60 vars in 1140 bytes} [Thu Jun 23 19:47:31 2022] POST /identity/v3/projects => generated 298 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 19:47:33.786478 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df480881-3504-44e6-9ff8-5f21411a6d52 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:33.787020 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df480881-3504-44e6-9ff8-5f21411a6d52 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:33.787020 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df480881-3504-44e6-9ff8-5f21411a6d52 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:33.788423 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 32/58] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:33.793926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a66a10-0a9f-4a61-973c-6df7e67904d1 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:33.794330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a66a10-0a9f-4a61-973c-6df7e67904d1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:33.794492 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a66a10-0a9f-4a61-973c-6df7e67904d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:33.810721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-72a66a10-0a9f-4a61-973c-6df7e67904d1 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:33.811910 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-72a66a10-0a9f-4a61-973c-6df7e67904d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:33.875810 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 27/59] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:33.899941 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df480881-3504-44e6-9ff8-5f21411a6d52 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:33.997631 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df480881-3504-44e6-9ff8-5f21411a6d52 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:34.030535 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498a0f3-f96f-41e3-90a4-8135a1e312da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:34.031309 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498a0f3-f96f-41e3-90a4-8135a1e312da None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:34.031595 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498a0f3-f96f-41e3-90a4-8135a1e312da None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:34.031872 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498a0f3-f96f-41e3-90a4-8135a1e312da None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:34.033064 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498a0f3-f96f-41e3-90a4-8135a1e312da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:34.034517 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498a0f3-f96f-41e3-90a4-8135a1e312da None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:34.036098 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 33/60] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:47:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:34.041911 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a66a10-0a9f-4a61-973c-6df7e67904d1 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:34.130739 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:34.139620 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a66a10-0a9f-4a61-973c-6df7e67904d1 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:34.175435 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300bb5cb-a82a-4ae1-8f51-ad558d993dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:34.176111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bb5cb-a82a-4ae1-8f51-ad558d993dd3 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:34.176422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bb5cb-a82a-4ae1-8f51-ad558d993dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:34.176701 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bb5cb-a82a-4ae1-8f51-ad558d993dd3 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:34.177422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300bb5cb-a82a-4ae1-8f51-ad558d993dd3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:34.178888 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300bb5cb-a82a-4ae1-8f51-ad558d993dd3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:34.291589 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 28/61] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jun 23 19:47:34 2022] POST /identity/v3/users => generated 275 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 19:47:37.484909 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc9b1e-9fe1-44d0-bc31-1f28a523949b None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:37.485954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc9b1e-9fe1-44d0-bc31-1f28a523949b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:37.486504 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc9b1e-9fe1-44d0-bc31-1f28a523949b None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:37.488105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 34/62] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:37.494019 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db773f0f-11a6-4aaf-a462-0a8002050257 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:37.494876 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db773f0f-11a6-4aaf-a462-0a8002050257 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:37.495368 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db773f0f-11a6-4aaf-a462-0a8002050257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:37.517304 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-db773f0f-11a6-4aaf-a462-0a8002050257 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:37.518848 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-db773f0f-11a6-4aaf-a462-0a8002050257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:37.603431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 29/63] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:37.624978 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f0d06-3e5f-4338-902c-88286950e487 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:37.625563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f0d06-3e5f-4338-902c-88286950e487 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:37.626039 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f0d06-3e5f-4338-902c-88286950e487 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:37.627677 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 35/64] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:37.633918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a82d1-ecc5-4148-8b26-76c8d39da46a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:37.634463 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a82d1-ecc5-4148-8b26-76c8d39da46a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:37.634948 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a82d1-ecc5-4148-8b26-76c8d39da46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:37.639237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f0d06-3e5f-4338-902c-88286950e487 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:37.655503 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5e1a82d1-ecc5-4148-8b26-76c8d39da46a None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:37.657049 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5e1a82d1-ecc5-4148-8b26-76c8d39da46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:37.696657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:37.750960 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 30/65] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:37.775947 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a82d1-ecc5-4148-8b26-76c8d39da46a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:37.784993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f0d06-3e5f-4338-902c-88286950e487 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:37.789161 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:37.853902 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1843adeb-0780-4b7f-83d9-aa4ba4b7200c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:37.854701 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1843adeb-0780-4b7f-83d9-aa4ba4b7200c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:37.854981 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1843adeb-0780-4b7f-83d9-aa4ba4b7200c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:37.855252 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1843adeb-0780-4b7f-83d9-aa4ba4b7200c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:37.866351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1843adeb-0780-4b7f-83d9-aa4ba4b7200c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:47:37.874997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-1843adeb-0780-4b7f-83d9-aa4ba4b7200c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:47:37.874997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:47:37.874997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:47:37.874997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:47:37.874997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:47:37.874997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:47:37.874997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:47:37.874997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:47:37.874997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:47:37.874997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:47:37.874997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:47:37.874997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:47:37.874997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:47:37.874997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:47:37.874997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:47:37.874997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:47:37.875881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:47:37.875881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:47:37.875881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:47:37.875881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:47:37.875881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:47:37.875881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:47:37.875881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:47:37.875881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:47:37.875881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:47:37.875881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:47:37.875881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:47:37.875881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:47:37.875881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:47:37.875881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:47:37.875881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:47:37.875881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:47:37.876727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:47:37.876727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:47:37.876727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:47:37.876727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:47:37.876727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:47:37.876727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:47:37.876727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:47:37.876727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:47:37.876727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:47:37.876727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:47:37.876727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:47:37.876727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:47:37.876727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 36/66] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:47:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:47:37.877479 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2417546-dea7-4462-bbda-a29817ece896 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:37.877926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2417546-dea7-4462-bbda-a29817ece896 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:37.878427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2417546-dea7-4462-bbda-a29817ece896 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:37.881290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 37/67] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:37.882297 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2417546-dea7-4462-bbda-a29817ece896 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:37.894292 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a82d1-ecc5-4148-8b26-76c8d39da46a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:37.929922 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78845911-1bb2-454f-bc8c-5c84738790f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:37.930958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78845911-1bb2-454f-bc8c-5c84738790f3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:37.931415 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78845911-1bb2-454f-bc8c-5c84738790f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:37.931686 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78845911-1bb2-454f-bc8c-5c84738790f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:37.942608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78845911-1bb2-454f-bc8c-5c84738790f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:47:37.950495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-78845911-1bb2-454f-bc8c-5c84738790f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:47:37.950495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:47:37.950495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:47:37.950495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:47:37.950495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:47:37.950495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:47:37.950495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:47:37.950495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:47:37.950495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:47:37.950495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:47:37.950495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:47:37.950495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:47:37.950495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:47:37.950495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:47:37.950495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:47:37.950495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:47:37.951540 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:47:37.951540 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:47:37.951540 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:47:37.951540 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:47:37.951540 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:47:37.951540 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:47:37.951540 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:47:37.951540 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:47:37.951540 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:47:37.951540 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:47:37.951540 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:47:37.951540 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:47:37.951540 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:47:37.951540 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:47:37.951540 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:47:37.951540 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:47:37.952524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:47:37.952524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:47:37.952524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:47:37.952524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:47:37.952524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:47:37.952524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:47:37.952524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:47:37.952524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:47:37.952524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:47:37.952524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:47:37.952524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:47:37.952524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:47:37.954530 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 31/68] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:47:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:47:37.956010 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e975f3a-536d-4bdf-afe2-7b2da43632b1 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:37.956467 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e975f3a-536d-4bdf-afe2-7b2da43632b1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:37.956893 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e975f3a-536d-4bdf-afe2-7b2da43632b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:37.973860 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-0e975f3a-536d-4bdf-afe2-7b2da43632b1 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:37.975269 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0e975f3a-536d-4bdf-afe2-7b2da43632b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:38.001635 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:38.012553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2417546-dea7-4462-bbda-a29817ece896 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:38.043580 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:38.048950 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6825f-081c-41ce-b753-cb3bfa42e573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:38.049102 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 32/69] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:38.049420 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6825f-081c-41ce-b753-cb3bfa42e573 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:38.049706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6825f-081c-41ce-b753-cb3bfa42e573 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:38.049981 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6825f-081c-41ce-b753-cb3bfa42e573 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:38.050739 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e6825f-081c-41ce-b753-cb3bfa42e573 None admin] RBAC: Adding query filter params (name=member) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:38.051047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e6825f-081c-41ce-b753-cb3bfa42e573 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:38.052475 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e6825f-081c-41ce-b753-cb3bfa42e573 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:38.058855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e975f3a-536d-4bdf-afe2-7b2da43632b1 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:38.063703 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 38/70] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jun 23 19:47:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:38.065192 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33daa8c8-c227-4302-9e74-8550c171c97b None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:38.065716 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33daa8c8-c227-4302-9e74-8550c171c97b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:38.066189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33daa8c8-c227-4302-9e74-8550c171c97b None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:38.067643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 39/71] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:38.072578 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33daa8c8-c227-4302-9e74-8550c171c97b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:38.176403 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e975f3a-536d-4bdf-afe2-7b2da43632b1 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:38.181584 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33daa8c8-c227-4302-9e74-8550c171c97b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:38.214311 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b21432-5954-4385-b9c8-8d1656c3fc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:38.214981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b21432-5954-4385-b9c8-8d1656c3fc6e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:38.215272 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b21432-5954-4385-b9c8-8d1656c3fc6e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:38.215540 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b21432-5954-4385-b9c8-8d1656c3fc6e None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:38.216290 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b21432-5954-4385-b9c8-8d1656c3fc6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:38.216739 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b21432-5954-4385-b9c8-8d1656c3fc6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:38.218268 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b21432-5954-4385-b9c8-8d1656c3fc6e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:38.220347 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f6f4f-71d4-4beb-8f06-0a15d44bf3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:38.221226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f6f4f-71d4-4beb-8f06-0a15d44bf3aa None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:38.221509 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f6f4f-71d4-4beb-8f06-0a15d44bf3aa None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:38.221785 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f6f4f-71d4-4beb-8f06-0a15d44bf3aa None admin] PATH_INFO: `/v3/users/fa6694f71dec4568a4b7724c3570eff4` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:38.229041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 33/72] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:47:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:38.229041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb4d69a-2657-4e11-87f1-17c8010f8641 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:38.229376 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb4d69a-2657-4e11-87f1-17c8010f8641 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:38.229641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb4d69a-2657-4e11-87f1-17c8010f8641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:38.242219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f6f4f-71d4-4beb-8f06-0a15d44bf3aa None admin] RBAC: Authorizing `identity:get_user(user_id=fa6694f71dec4568a4b7724c3570eff4)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:38.243725 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f6f4f-71d4-4beb-8f06-0a15d44bf3aa None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:38.245522 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 40/73] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:47:38 2022] GET /identity/v3/users/fa6694f71dec4568a4b7724c3570eff4 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:38.246280 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f6f4f-71d4-4beb-8f06-0a15d44bf3aa None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:38.248773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-fcb4d69a-2657-4e11-87f1-17c8010f8641 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:38.250229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fcb4d69a-2657-4e11-87f1-17c8010f8641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:38.261368 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:38.313273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:38.325081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 34/74] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:38.326390 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3f467-8436-4dd4-a6a8-3d43bd796494 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:38.326675 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3f467-8436-4dd4-a6a8-3d43bd796494 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:38.326948 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3f467-8436-4dd4-a6a8-3d43bd796494 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:38.328361 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 35/75] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:38.329096 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf3f467-8436-4dd4-a6a8-3d43bd796494 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:38.364104 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f6f4f-71d4-4beb-8f06-0a15d44bf3aa None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:38.403027 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9bde21-471c-40fc-acda-1542bb287649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:38.406429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bde21-471c-40fc-acda-1542bb287649 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:38.406429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bde21-471c-40fc-acda-1542bb287649 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:38.406429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bde21-471c-40fc-acda-1542bb287649 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:38.415975 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9bde21-471c-40fc-acda-1542bb287649 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:47:38.427465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-ff9bde21-471c-40fc-acda-1542bb287649 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 19:47:38.427465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:47:38.427465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:47:38.427465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:47:38.427465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:47:38.427465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:47:38.427465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:47:38.427465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:47:38.427465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:47:38.427465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:47:38.427465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:47:38.427465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:47:38.427465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:47:38.427465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:47:38.427465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:47:38.427465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:47:38.428358 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:47:38.428358 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:47:38.428358 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:47:38.428358 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:47:38.428358 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:47:38.428358 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:47:38.428358 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:47:38.428358 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:47:38.428358 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:47:38.428358 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:47:38.428358 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:47:38.428358 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:47:38.428358 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:47:38.428358 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:47:38.428358 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:47:38.428358 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:47:38.429192 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:47:38.429192 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:47:38.429192 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:47:38.429192 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:47:38.429192 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:47:38.429192 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:47:38.429192 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:47:38.429192 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:47:38.429192 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:47:38.429192 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:47:38.429192 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 19:47:38.429192 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:47:38.429192 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 41/76] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jun 23 19:47:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:47:38.429864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9bde21-471c-40fc-acda-1542bb287649 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:38.445987 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf3f467-8436-4dd4-a6a8-3d43bd796494 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:38.488988 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc920ab-1d88-463f-856c-4b06b2cc57de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:38.489944 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc920ab-1d88-463f-856c-4b06b2cc57de None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:38.490233 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc920ab-1d88-463f-856c-4b06b2cc57de None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:38.490512 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc920ab-1d88-463f-856c-4b06b2cc57de None admin] PATH_INFO: `/v3/users/5060168fc25744178d345904afb6c333` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:38.492292 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc920ab-1d88-463f-856c-4b06b2cc57de None admin] RBAC: Authorizing `identity:get_user(user_id=5060168fc25744178d345904afb6c333)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:38.495076 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc920ab-1d88-463f-856c-4b06b2cc57de None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:38.498894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 36/77] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:47:38 2022] GET /identity/v3/users/5060168fc25744178d345904afb6c333 => generated 247 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:38.502266 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5641d2-a259-4861-a18f-af07ccd4df8c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:38.502566 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5641d2-a259-4861-a18f-af07ccd4df8c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:38.502853 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5641d2-a259-4861-a18f-af07ccd4df8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:38.506050 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:38.525498 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-da5641d2-a259-4861-a18f-af07ccd4df8c None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:38.525498 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-da5641d2-a259-4861-a18f-af07ccd4df8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:38.537954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9bde21-471c-40fc-acda-1542bb287649 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:38.577517 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c85efc-7405-4936-a2e0-e8c15f36fc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:38.578182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c85efc-7405-4936-a2e0-e8c15f36fc20 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:38.578469 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c85efc-7405-4936-a2e0-e8c15f36fc20 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:38.578745 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c85efc-7405-4936-a2e0-e8c15f36fc20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:38.586725 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:38.604137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 37/78] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:38.605187 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5641d2-a259-4861-a18f-af07ccd4df8c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:38.658197 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c85efc-7405-4936-a2e0-e8c15f36fc20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:38.660206 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c85efc-7405-4936-a2e0-e8c15f36fc20 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:38.716502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5641d2-a259-4861-a18f-af07ccd4df8c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:38.733483 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:38.754890 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a8e83-be46-448c-8b8a-2116559afa60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:38.756247 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a8e83-be46-448c-8b8a-2116559afa60 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:38.756792 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a8e83-be46-448c-8b8a-2116559afa60 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:38.757284 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a8e83-be46-448c-8b8a-2116559afa60 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:38.765254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 42/79] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:47:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:47:38.766103 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c85efc-7405-4936-a2e0-e8c15f36fc20 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:38.769125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a8e83-be46-448c-8b8a-2116559afa60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:47:38.778069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-a56a8e83-be46-448c-8b8a-2116559afa60 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:47:38.778069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:47:38.778069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:47:38.778069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:47:38.778069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:47:38.778069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:47:38.778069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:47:38.778069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:47:38.778069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:47:38.778069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:47:38.778069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:47:38.778069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:47:38.778069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:47:38.778069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:47:38.778069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:47:38.778069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:47:38.779028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:47:38.779028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:47:38.779028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:47:38.779028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:47:38.779028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:47:38.779028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:47:38.779028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:47:38.779028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:47:38.779028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:47:38.779028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:47:38.779028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:47:38.779028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:47:38.779028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:47:38.779028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:47:38.779028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:47:38.779028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:47:38.779789 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:47:38.779789 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:47:38.779789 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:47:38.779789 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:47:38.779789 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:47:38.779789 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:47:38.779789 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:47:38.779789 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:47:38.779789 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:47:38.779789 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:47:38.779789 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:47:38.779789 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:47:38.779789 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 38/80] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:47:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:47:38.780465 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a8e83-be46-448c-8b8a-2116559afa60 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:38.835646 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:38.878652 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c85efc-7405-4936-a2e0-e8c15f36fc20 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:38.886530 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a8e83-be46-448c-8b8a-2116559afa60 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:38.912241 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdda979-11e2-41e1-ba4f-40cd6a79c331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:38.912930 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdda979-11e2-41e1-ba4f-40cd6a79c331 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:38.913216 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdda979-11e2-41e1-ba4f-40cd6a79c331 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:38.913493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdda979-11e2-41e1-ba4f-40cd6a79c331 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:38.914261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdda979-11e2-41e1-ba4f-40cd6a79c331 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:38.914574 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdda979-11e2-41e1-ba4f-40cd6a79c331 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:38.915961 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdda979-11e2-41e1-ba4f-40cd6a79c331 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:38.919781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585998c-f674-4ab3-8364-e0b77ab30423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:38.920457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585998c-f674-4ab3-8364-e0b77ab30423 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:38.920756 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585998c-f674-4ab3-8364-e0b77ab30423 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:38.921038 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585998c-f674-4ab3-8364-e0b77ab30423 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:38.923815 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 43/81] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:47:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:38.924632 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdda979-11e2-41e1-ba4f-40cd6a79c331 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:38.956183 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:38.987205 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d585998c-f674-4ab3-8364-e0b77ab30423 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:38.988778 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d585998c-f674-4ab3-8364-e0b77ab30423 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:39.022284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdda979-11e2-41e1-ba4f-40cd6a79c331 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:39.035945 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:39.055012 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc9f21-2cd0-4496-97c1-1b04c93445d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:39.055796 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc9f21-2cd0-4496-97c1-1b04c93445d6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:39.056082 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc9f21-2cd0-4496-97c1-1b04c93445d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:39.056410 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc9f21-2cd0-4496-97c1-1b04c93445d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:39.057695 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc9f21-2cd0-4496-97c1-1b04c93445d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:39.059217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc9f21-2cd0-4496-97c1-1b04c93445d6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:39.060866 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 44/82] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:47:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:39.061641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc9f21-2cd0-4496-97c1-1b04c93445d6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:39.087525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 39/83] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:47:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:47:39.088418 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585998c-f674-4ab3-8364-e0b77ab30423 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:39.170174 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:39.173021 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc9f21-2cd0-4496-97c1-1b04c93445d6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:39.185271 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585998c-f674-4ab3-8364-e0b77ab30423 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:39.207293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe5742-5607-4a0c-8617-ce0cbd03de5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:39.208159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe5742-5607-4a0c-8617-ce0cbd03de5c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:39.208478 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe5742-5607-4a0c-8617-ce0cbd03de5c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:39.208758 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe5742-5607-4a0c-8617-ce0cbd03de5c None admin] PATH_INFO: `/v3/projects/4c94dfce039f4ff29bec2f9831ec1748` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:39.210253 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe5742-5607-4a0c-8617-ce0cbd03de5c None admin] RBAC: Authorizing `identity:get_project(project_id=4c94dfce039f4ff29bec2f9831ec1748)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:39.211776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe5742-5607-4a0c-8617-ce0cbd03de5c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:39.213536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 45/84] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:47:39 2022] GET /identity/v3/projects/4c94dfce039f4ff29bec2f9831ec1748 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:39.214303 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe5742-5607-4a0c-8617-ce0cbd03de5c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:39.220959 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc20356-e969-4c2e-80a0-f9c0fb420319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:39.221630 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc20356-e969-4c2e-80a0-f9c0fb420319 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:39.221917 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc20356-e969-4c2e-80a0-f9c0fb420319 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:39.222195 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc20356-e969-4c2e-80a0-f9c0fb420319 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:39.222950 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc20356-e969-4c2e-80a0-f9c0fb420319 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:39.223258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc20356-e969-4c2e-80a0-f9c0fb420319 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:39.224676 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc20356-e969-4c2e-80a0-f9c0fb420319 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:39.234684 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 40/85] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jun 23 19:47:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:39.234684 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc20356-e969-4c2e-80a0-f9c0fb420319 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:39.248540 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:39.316762 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe5742-5607-4a0c-8617-ce0cbd03de5c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:39.334223 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc20356-e969-4c2e-80a0-f9c0fb420319 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:39.348987 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123b1e1-032d-4a40-a11d-9ac1d98156b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:39.349857 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123b1e1-032d-4a40-a11d-9ac1d98156b1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:39.350146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123b1e1-032d-4a40-a11d-9ac1d98156b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:39.350428 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123b1e1-032d-4a40-a11d-9ac1d98156b1 None admin] PATH_INFO: `/v3/users/fa6694f71dec4568a4b7724c3570eff4` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:39.352191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1123b1e1-032d-4a40-a11d-9ac1d98156b1 None admin] RBAC: Authorizing `identity:get_user(user_id=fa6694f71dec4568a4b7724c3570eff4)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:39.353714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1123b1e1-032d-4a40-a11d-9ac1d98156b1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:39.355561 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 46/86] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:47:39 2022] GET /identity/v3/users/fa6694f71dec4568a4b7724c3570eff4 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:39.356123 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123b1e1-032d-4a40-a11d-9ac1d98156b1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:39.371486 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e392cd0-afa7-4618-9a13-deb7618a12e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:39.372155 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e392cd0-afa7-4618-9a13-deb7618a12e2 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:39.372471 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e392cd0-afa7-4618-9a13-deb7618a12e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:39.372749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e392cd0-afa7-4618-9a13-deb7618a12e2 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:39.373456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e392cd0-afa7-4618-9a13-deb7618a12e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:39.374904 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e392cd0-afa7-4618-9a13-deb7618a12e2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:39.399900 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:39.469326 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123b1e1-032d-4a40-a11d-9ac1d98156b1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:39.487919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 41/87] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jun 23 19:47:39 2022] POST /identity/v3/users => generated 289 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:47:39.491587 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e392cd0-afa7-4618-9a13-deb7618a12e2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:39.505208 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c91684-fefc-4092-96bb-5bf8aaf232bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:39.506158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c91684-fefc-4092-96bb-5bf8aaf232bf None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:39.506695 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c91684-fefc-4092-96bb-5bf8aaf232bf None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:39.506809 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c91684-fefc-4092-96bb-5bf8aaf232bf None admin] PATH_INFO: `/v3/projects/4c94dfce039f4ff29bec2f9831ec1748` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:39.508293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c91684-fefc-4092-96bb-5bf8aaf232bf None admin] RBAC: Authorizing `identity:get_project(project_id=4c94dfce039f4ff29bec2f9831ec1748)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:39.510807 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c91684-fefc-4092-96bb-5bf8aaf232bf None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:39.518130 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 47/88] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:47:39 2022] GET /identity/v3/projects/4c94dfce039f4ff29bec2f9831ec1748 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:39.518130 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c91684-fefc-4092-96bb-5bf8aaf232bf None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:39.575315 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:39.619337 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e392cd0-afa7-4618-9a13-deb7618a12e2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:39.628451 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c91684-fefc-4092-96bb-5bf8aaf232bf None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:39.630886 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:39.659329 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c1c385-ebf4-4798-a671-d49fd829fb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:39.660079 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1c385-ebf4-4798-a671-d49fd829fb8f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:39.660426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1c385-ebf4-4798-a671-d49fd829fb8f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:39.660706 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1c385-ebf4-4798-a671-d49fd829fb8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:39.661555 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c1c385-ebf4-4798-a671-d49fd829fb8f None admin] RBAC: Adding query filter params (role.id=860491bfbb574772a7a213d6a25962e0, scope.project.id=4c94dfce039f4ff29bec2f9831ec1748, user.id=fa6694f71dec4568a4b7724c3570eff4) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:39.661875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c1c385-ebf4-4798-a671-d49fd829fb8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:39.663348 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c1c385-ebf4-4798-a671-d49fd829fb8f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:39.665245 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa3a99-476e-44ac-9fb4-442f0110d613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:39.666123 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa3a99-476e-44ac-9fb4-442f0110d613 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:39.666410 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa3a99-476e-44ac-9fb4-442f0110d613 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:39.666690 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa3a99-476e-44ac-9fb4-442f0110d613 None admin] PATH_INFO: `/v3/users/fa6694f71dec4568a4b7724c3570eff4` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:39.667899 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa3a99-476e-44ac-9fb4-442f0110d613 None admin] RBAC: Authorizing `identity:get_user(user_id=fa6694f71dec4568a4b7724c3570eff4)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:39.669444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa3a99-476e-44ac-9fb4-442f0110d613 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:39.671404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 48/89] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:47:39 2022] GET /identity/v3/users/fa6694f71dec4568a4b7724c3570eff4 => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:39.672258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa3a99-476e-44ac-9fb4-442f0110d613 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:39.673465 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 42/90] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:47:39 2022] GET /identity/v3/role_assignments?user.id=fa6694f71dec4568a4b7724c3570eff4&scope.project.id=4c94dfce039f4ff29bec2f9831ec1748&role.id=860491bfbb574772a7a213d6a25962e0 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:39.674282 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c1c385-ebf4-4798-a671-d49fd829fb8f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:39.818498 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa3a99-476e-44ac-9fb4-442f0110d613 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:39.821886 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c1c385-ebf4-4798-a671-d49fd829fb8f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:39.865923 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:39.871218 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee97c61-f4ea-4d1b-bb4e-f366d3f5baa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:39.871882 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee97c61-f4ea-4d1b-bb4e-f366d3f5baa2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:39.872164 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee97c61-f4ea-4d1b-bb4e-f366d3f5baa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:39.872515 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee97c61-f4ea-4d1b-bb4e-f366d3f5baa2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:39.873199 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab47fc-d3cb-4250-bc00-66090bba2999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:39.873829 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab47fc-d3cb-4250-bc00-66090bba2999 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:39.874122 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab47fc-d3cb-4250-bc00-66090bba2999 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:39.874395 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab47fc-d3cb-4250-bc00-66090bba2999 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:39.875245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab47fc-d3cb-4250-bc00-66090bba2999 None admin] RBAC: Adding query filter params (role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb, scope.project.id=4c94dfce039f4ff29bec2f9831ec1748, user.id=5060168fc25744178d345904afb6c333) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:39.875562 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab47fc-d3cb-4250-bc00-66090bba2999 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:39.878364 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab47fc-d3cb-4250-bc00-66090bba2999 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:39.884039 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 43/91] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:47:39 2022] GET /identity/v3/role_assignments?user.id=5060168fc25744178d345904afb6c333&scope.project.id=4c94dfce039f4ff29bec2f9831ec1748&role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:39.884748 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab47fc-d3cb-4250-bc00-66090bba2999 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:39.929076 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:39.963591 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee97c61-f4ea-4d1b-bb4e-f366d3f5baa2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:39.966230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee97c61-f4ea-4d1b-bb4e-f366d3f5baa2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:40.025547 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab47fc-d3cb-4250-bc00-66090bba2999 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:40.073974 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74fd77-0847-4288-af18-6462cdbfd6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:40.074653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74fd77-0847-4288-af18-6462cdbfd6cf None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:40.074942 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74fd77-0847-4288-af18-6462cdbfd6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:40.075224 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74fd77-0847-4288-af18-6462cdbfd6cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:40.080045 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 49/92] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:47:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:47:40.085245 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee97c61-f4ea-4d1b-bb4e-f366d3f5baa2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:40.162592 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:40.167251 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:40.173816 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74fd77-0847-4288-af18-6462cdbfd6cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:40.178454 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74fd77-0847-4288-af18-6462cdbfd6cf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:40.200282 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee97c61-f4ea-4d1b-bb4e-f366d3f5baa2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:40.246029 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f5f579-f5cb-473a-ba9a-3dcbff31b89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:40.247129 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f5f579-f5cb-473a-ba9a-3dcbff31b89e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:40.247563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f5f579-f5cb-473a-ba9a-3dcbff31b89e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:40.247985 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f5f579-f5cb-473a-ba9a-3dcbff31b89e None admin] PATH_INFO: `/v3/projects/4c94dfce039f4ff29bec2f9831ec1748` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:40.249460 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f5f579-f5cb-473a-ba9a-3dcbff31b89e None admin] RBAC: Authorizing `identity:get_project(project_id=4c94dfce039f4ff29bec2f9831ec1748)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:40.251167 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f5f579-f5cb-473a-ba9a-3dcbff31b89e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:40.253311 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 50/93] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:47:40 2022] GET /identity/v3/projects/4c94dfce039f4ff29bec2f9831ec1748 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:47:40.261616 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f5f579-f5cb-473a-ba9a-3dcbff31b89e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:40.303551 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 44/94] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:47:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:47:40.310110 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74fd77-0847-4288-af18-6462cdbfd6cf None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:40.479055 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f5f579-f5cb-473a-ba9a-3dcbff31b89e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:40.532660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:40.541886 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957ed12-1c26-44ad-8f78-0df6428802e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:40.542550 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957ed12-1c26-44ad-8f78-0df6428802e6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:40.542868 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957ed12-1c26-44ad-8f78-0df6428802e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:40.543145 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a957ed12-1c26-44ad-8f78-0df6428802e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:40.543990 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a957ed12-1c26-44ad-8f78-0df6428802e6 None admin] RBAC: Adding query filter params (role.id=03e4cfc753754643ae08fa909647cfe3, scope.project.id=4c94dfce039f4ff29bec2f9831ec1748, user.id=fa6694f71dec4568a4b7724c3570eff4) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:40.544317 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a957ed12-1c26-44ad-8f78-0df6428802e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:40.549115 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a957ed12-1c26-44ad-8f78-0df6428802e6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:40.551118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74fd77-0847-4288-af18-6462cdbfd6cf None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:40.557345 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 51/95] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:47:40 2022] GET /identity/v3/role_assignments?user.id=fa6694f71dec4568a4b7724c3570eff4&scope.project.id=4c94dfce039f4ff29bec2f9831ec1748&role.id=03e4cfc753754643ae08fa909647cfe3 => generated 259 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:40.619319 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d5d70-8699-49a5-acb1-50b27de5efc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:40.620192 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d5d70-8699-49a5-acb1-50b27de5efc3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:40.620502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d5d70-8699-49a5-acb1-50b27de5efc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:40.620783 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d5d70-8699-49a5-acb1-50b27de5efc3 None admin] PATH_INFO: `/v3/projects/5cfe3739530c48c29d910951147203ea` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:40.625238 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d5d70-8699-49a5-acb1-50b27de5efc3 None admin] RBAC: Authorizing `identity:get_project(project_id=5cfe3739530c48c29d910951147203ea)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:40.626759 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d5d70-8699-49a5-acb1-50b27de5efc3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:40.628627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 45/96] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:47:40 2022] GET /identity/v3/projects/5cfe3739530c48c29d910951147203ea => generated 312 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:40.635104 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957ed12-1c26-44ad-8f78-0df6428802e6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:40.651954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:40.811982 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a957ed12-1c26-44ad-8f78-0df6428802e6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:40.861203 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd555cae-670e-45e2-a15f-202a28cee560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:40.861864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd555cae-670e-45e2-a15f-202a28cee560 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:40.862148 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd555cae-670e-45e2-a15f-202a28cee560 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:40.862424 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd555cae-670e-45e2-a15f-202a28cee560 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:40.863264 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd555cae-670e-45e2-a15f-202a28cee560 None admin] RBAC: Adding query filter params (role.id=860491bfbb574772a7a213d6a25962e0, scope.project.id=5cfe3739530c48c29d910951147203ea, user.id=fa6694f71dec4568a4b7724c3570eff4) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:40.863579 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd555cae-670e-45e2-a15f-202a28cee560 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:40.865125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd555cae-670e-45e2-a15f-202a28cee560 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:40.876916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 52/97] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:47:40 2022] GET /identity/v3/role_assignments?user.id=fa6694f71dec4568a4b7724c3570eff4&scope.project.id=5cfe3739530c48c29d910951147203ea&role.id=860491bfbb574772a7a213d6a25962e0 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:42.451634 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71eeb2d-ada1-4397-a188-9860b3ab555a None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:42.452693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71eeb2d-ada1-4397-a188-9860b3ab555a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:42.453217 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71eeb2d-ada1-4397-a188-9860b3ab555a None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:42.454826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 46/98] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:42.460546 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f9d11-91e5-4124-9993-b8acb76a3939 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:42.461444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f9d11-91e5-4124-9993-b8acb76a3939 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:42.461921 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f9d11-91e5-4124-9993-b8acb76a3939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:42.484800 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-7f5f9d11-91e5-4124-9993-b8acb76a3939 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:42.486384 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7f5f9d11-91e5-4124-9993-b8acb76a3939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:42.569024 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:42.574921 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 53/99] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:42.575885 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d822f8-4281-445b-b179-277d3b5642ae None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:42.576178 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d822f8-4281-445b-b179-277d3b5642ae None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:42.576495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d822f8-4281-445b-b179-277d3b5642ae None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:42.577850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 47/100] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:42.597382 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5a0da-8057-48fe-a7b1-1fd0c78abd52 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:42.597665 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5a0da-8057-48fe-a7b1-1fd0c78abd52 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:42.597951 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5a0da-8057-48fe-a7b1-1fd0c78abd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:42.605488 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d822f8-4281-445b-b179-277d3b5642ae None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:42.622467 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-53d5a0da-8057-48fe-a7b1-1fd0c78abd52 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:42.624395 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-53d5a0da-8057-48fe-a7b1-1fd0c78abd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:42.704412 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 54/101] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:42.724248 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5a0da-8057-48fe-a7b1-1fd0c78abd52 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:42.793068 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d822f8-4281-445b-b179-277d3b5642ae None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:42.867103 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b69758-a7df-43bd-a3ef-c876d00d69bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:42.868015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b69758-a7df-43bd-a3ef-c876d00d69bf None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:42.868321 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b69758-a7df-43bd-a3ef-c876d00d69bf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:42.868627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b69758-a7df-43bd-a3ef-c876d00d69bf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:42.874589 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5a0da-8057-48fe-a7b1-1fd0c78abd52 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:42.893222 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b69758-a7df-43bd-a3ef-c876d00d69bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:47:42.901868 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-d3b69758-a7df-43bd-a3ef-c876d00d69bf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 19:47:42.901868 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:47:42.901868 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:47:42.901868 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:47:42.901868 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:47:42.901868 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:47:42.901868 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:47:42.901868 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:47:42.901868 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:47:42.901868 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:47:42.901868 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:47:42.901868 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:47:42.901868 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:47:42.901868 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:47:42.901868 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:47:42.901868 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:47:42.902727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:47:42.902727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:47:42.902727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:47:42.902727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:47:42.902727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:47:42.902727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:47:42.902727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:47:42.902727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:47:42.902727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:47:42.902727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:47:42.902727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:47:42.902727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:47:42.902727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:47:42.902727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:47:42.902727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:47:42.902727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:47:42.903460 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:47:42.903460 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:47:42.903460 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:47:42.903460 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:47:42.903460 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:47:42.903460 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:47:42.903460 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:47:42.903460 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:47:42.903460 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:47:42.903460 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:47:42.903460 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 19:47:42.903460 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:47:42.903460 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 48/102] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:47:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:47:42.914224 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b69758-a7df-43bd-a3ef-c876d00d69bf None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:42.933263 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:42.945472 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d823b39-6f1d-4768-91d1-d5b92cb86e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:42.946518 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d823b39-6f1d-4768-91d1-d5b92cb86e68 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:42.951871 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d823b39-6f1d-4768-91d1-d5b92cb86e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:42.951871 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d823b39-6f1d-4768-91d1-d5b92cb86e68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:42.954020 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d823b39-6f1d-4768-91d1-d5b92cb86e68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:42.956555 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d823b39-6f1d-4768-91d1-d5b92cb86e68 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:42.958641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 55/103] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:47:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:47:42.965927 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d823b39-6f1d-4768-91d1-d5b92cb86e68 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:42.980139 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:43.110133 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b69758-a7df-43bd-a3ef-c876d00d69bf None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:43.141719 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d823b39-6f1d-4768-91d1-d5b92cb86e68 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:43.183330 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9739cb4-cff1-4535-8c2e-6c14d5e8576e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:43.184003 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9739cb4-cff1-4535-8c2e-6c14d5e8576e None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:43.184291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9739cb4-cff1-4535-8c2e-6c14d5e8576e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:43.184600 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9739cb4-cff1-4535-8c2e-6c14d5e8576e None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:43.185476 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9739cb4-cff1-4535-8c2e-6c14d5e8576e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:43.187175 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9739cb4-cff1-4535-8c2e-6c14d5e8576e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:43.197636 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abbd366-130e-4f2d-bfed-e8b703b5715b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:43.198668 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abbd366-130e-4f2d-bfed-e8b703b5715b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:43.199143 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abbd366-130e-4f2d-bfed-e8b703b5715b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:43.199603 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abbd366-130e-4f2d-bfed-e8b703b5715b None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:43.200771 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abbd366-130e-4f2d-bfed-e8b703b5715b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:43.201272 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abbd366-130e-4f2d-bfed-e8b703b5715b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:43.202841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abbd366-130e-4f2d-bfed-e8b703b5715b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:43.216886 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 49/104] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jun 23 19:47:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:43.218318 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c679a-2364-41b8-b754-517d7a9650fd None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:43.218622 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c679a-2364-41b8-b754-517d7a9650fd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:43.218901 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c679a-2364-41b8-b754-517d7a9650fd None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:43.220401 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 50/105] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:43.225709 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c679a-2364-41b8-b754-517d7a9650fd None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:43.303232 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 56/106] 10.222.0.44 () {60 vars in 1141 bytes} [Thu Jun 23 19:47:42 2022] POST /identity/v3/projects => generated 302 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:47:43.304695 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40141379-0a68-43e4-ae0d-c9dce186dcb0 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:43.304993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40141379-0a68-43e4-ae0d-c9dce186dcb0 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:43.305274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40141379-0a68-43e4-ae0d-c9dce186dcb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:43.343983 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-40141379-0a68-43e4-ae0d-c9dce186dcb0 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:43.345236 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-40141379-0a68-43e4-ae0d-c9dce186dcb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:43.372767 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:43.392492 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c679a-2364-41b8-b754-517d7a9650fd None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:43.519893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:43.548723 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 57/107] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:43.550351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c25429-72f9-40c4-b169-db1155c33183 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:43.550831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c25429-72f9-40c4-b169-db1155c33183 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:43.551294 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c25429-72f9-40c4-b169-db1155c33183 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:43.552793 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 58/108] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:43.557099 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e9f84-5e16-4f53-b40b-3d9a65bb2272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:43.557994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e9f84-5e16-4f53-b40b-3d9a65bb2272 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:43.558337 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e9f84-5e16-4f53-b40b-3d9a65bb2272 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:43.558744 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e9f84-5e16-4f53-b40b-3d9a65bb2272 None admin] PATH_INFO: `/v3/users/f16b644eff4d40d092a047b8dc388112` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:43.561165 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c356687-12f2-4f3e-8fb9-e1f3ef82d37a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:43.561683 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c356687-12f2-4f3e-8fb9-e1f3ef82d37a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:43.562152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c356687-12f2-4f3e-8fb9-e1f3ef82d37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:43.586631 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-4c356687-12f2-4f3e-8fb9-e1f3ef82d37a None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:43.588786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4c356687-12f2-4f3e-8fb9-e1f3ef82d37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:43.589258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e9f84-5e16-4f53-b40b-3d9a65bb2272 None admin] RBAC: Authorizing `identity:get_user(user_id=f16b644eff4d40d092a047b8dc388112)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:43.591910 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e9f84-5e16-4f53-b40b-3d9a65bb2272 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:43.593736 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 51/109] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:47:43 2022] GET /identity/v3/users/f16b644eff4d40d092a047b8dc388112 => generated 289 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:43.594983 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b642c-59af-451b-b42f-b17f32655a36 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:43.595322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b642c-59af-451b-b42f-b17f32655a36 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:43.595645 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b642c-59af-451b-b42f-b17f32655a36 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:43.597180 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 52/110] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:43.597893 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b642c-59af-451b-b42f-b17f32655a36 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:43.666316 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 59/111] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:43.667568 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b27e57-e03b-4eee-88eb-0c96d710affe None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:43.667864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b27e57-e03b-4eee-88eb-0c96d710affe None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:43.668209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b27e57-e03b-4eee-88eb-0c96d710affe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:43.697993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f7b27e57-e03b-4eee-88eb-0c96d710affe None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:43.699205 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f7b27e57-e03b-4eee-88eb-0c96d710affe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:43.747817 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b642c-59af-451b-b42f-b17f32655a36 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:43.769990 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 60/112] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:43.770950 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b27e57-e03b-4eee-88eb-0c96d710affe None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:43.790340 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cacd50-5c4c-41c2-afe4-ec629ac655e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:43.791137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cacd50-5c4c-41c2-afe4-ec629ac655e2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:43.791514 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cacd50-5c4c-41c2-afe4-ec629ac655e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:43.791840 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cacd50-5c4c-41c2-afe4-ec629ac655e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:43.803403 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cacd50-5c4c-41c2-afe4-ec629ac655e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:47:43.813342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-99cacd50-5c4c-41c2-afe4-ec629ac655e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:47:43.813342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:47:43.813342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:47:43.813342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:47:43.813342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:47:43.813342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:47:43.813342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:47:43.813342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:47:43.813342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:47:43.813342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:47:43.813342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:47:43.813342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:47:43.813342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:47:43.813342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:47:43.813342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:47:43.813342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:47:43.814166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:47:43.814166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:47:43.814166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:47:43.814166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:47:43.814166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:47:43.814166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:47:43.814166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:47:43.814166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:47:43.814166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:47:43.814166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:47:43.814166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:47:43.814166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:47:43.814166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:47:43.814166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:47:43.814166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:47:43.814166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:47:43.814918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:47:43.814918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:47:43.814918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:47:43.814918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:47:43.814918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:47:43.814918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:47:43.814918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:47:43.814918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:47:43.814918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:47:43.814918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:47:43.814918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:47:43.814918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:47:43.815493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:43.815560 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 53/113] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:47:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:47:43.815560 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cacd50-5c4c-41c2-afe4-ec629ac655e2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:43.818652 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:43.878654 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b27e57-e03b-4eee-88eb-0c96d710affe None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:43.919223 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd4311b-b703-468e-aa50-4161ab82e46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:43.919802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4311b-b703-468e-aa50-4161ab82e46a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:43.920091 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4311b-b703-468e-aa50-4161ab82e46a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:43.920399 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4311b-b703-468e-aa50-4161ab82e46a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:43.920559 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cacd50-5c4c-41c2-afe4-ec629ac655e2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:43.958860 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ff5d8-3827-4cb2-96b8-9be510035501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:43.959661 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ff5d8-3827-4cb2-96b8-9be510035501 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:43.960036 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ff5d8-3827-4cb2-96b8-9be510035501 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:43.960528 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ff5d8-3827-4cb2-96b8-9be510035501 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:43.972062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ff5d8-3827-4cb2-96b8-9be510035501 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:47:43.981186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-8f7ff5d8-3827-4cb2-96b8-9be510035501 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:47:43.981186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:47:43.981186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:47:43.981186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:47:43.981186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:47:43.981186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:47:43.981186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:47:43.981186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:47:43.981186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:47:43.981186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:47:43.981186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:47:43.981186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:47:43.981186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:47:43.981186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:47:43.981186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:47:43.981186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:47:43.982037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:47:43.982037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:47:43.982037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:47:43.982037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:47:43.982037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:47:43.982037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:47:43.982037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:47:43.982037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:47:43.982037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:47:43.982037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:47:43.982037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:47:43.982037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:47:43.982037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:47:43.982037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:47:43.982037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:47:43.982037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:47:43.983251 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:47:43.983251 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:47:43.983251 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:47:43.983251 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:47:43.983251 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:47:43.983251 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:47:43.983251 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:47:43.983251 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:47:43.983251 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:47:43.983251 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:47:43.983251 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:47:43.983251 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:47:43.983251 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 54/114] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:47:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:47:43.984074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a6085-61c1-4a29-a1cb-8e010479a925 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:43.984074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a6085-61c1-4a29-a1cb-8e010479a925 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:43.984369 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a6085-61c1-4a29-a1cb-8e010479a925 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:43.985759 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 55/115] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:43.990415 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655a6085-61c1-4a29-a1cb-8e010479a925 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:43.996661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd4311b-b703-468e-aa50-4161ab82e46a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:43.998190 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd4311b-b703-468e-aa50-4161ab82e46a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:44.036215 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:44.057774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:44.098375 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655a6085-61c1-4a29-a1cb-8e010479a925 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:44.106071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 61/116] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:47:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:47:44.106071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd4311b-b703-468e-aa50-4161ab82e46a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:44.137470 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce1a14-f8f8-419c-af3a-54ef86843ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:44.138444 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce1a14-f8f8-419c-af3a-54ef86843ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:44.138736 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce1a14-f8f8-419c-af3a-54ef86843ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:44.139015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce1a14-f8f8-419c-af3a-54ef86843ea6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:44.149688 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce1a14-f8f8-419c-af3a-54ef86843ea6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:47:44.157786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-32ce1a14-f8f8-419c-af3a-54ef86843ea6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 19:47:44.157786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:47:44.157786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:47:44.157786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:47:44.157786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:47:44.157786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:47:44.157786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:47:44.157786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:47:44.157786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:47:44.157786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:47:44.157786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:47:44.157786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:47:44.157786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:47:44.157786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:47:44.157786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:47:44.157786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:47:44.158642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:47:44.158642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:47:44.158642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:47:44.158642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:47:44.158642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:47:44.158642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:47:44.158642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:47:44.158642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:47:44.158642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:47:44.158642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:47:44.158642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:47:44.158642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:47:44.158642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:47:44.158642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:47:44.158642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:47:44.158642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:47:44.159424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:47:44.159424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:47:44.159424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:47:44.159424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:47:44.159424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:47:44.159424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:47:44.159424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:47:44.159424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:47:44.159424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:47:44.159424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:47:44.159424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 19:47:44.159424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:47:44.159424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 56/117] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jun 23 19:47:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:47:44.160039 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce1a14-f8f8-419c-af3a-54ef86843ea6 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:44.211600 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd4311b-b703-468e-aa50-4161ab82e46a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:44.251576 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cc509e-cb85-4908-9207-db071b9e2899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:44.252251 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc509e-cb85-4908-9207-db071b9e2899 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:44.252567 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc509e-cb85-4908-9207-db071b9e2899 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:44.252851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cc509e-cb85-4908-9207-db071b9e2899 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:44.253670 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc509e-cb85-4908-9207-db071b9e2899 None admin] RBAC: Adding query filter params (name=member) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:44.253987 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc509e-cb85-4908-9207-db071b9e2899 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:44.255444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cc509e-cb85-4908-9207-db071b9e2899 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:44.260645 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:44.268480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 62/118] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jun 23 19:47:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:44.268480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a40a5-6a11-438f-8ed3-a26502932f2a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:44.268480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a40a5-6a11-438f-8ed3-a26502932f2a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:44.268480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a40a5-6a11-438f-8ed3-a26502932f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:44.282614 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce1a14-f8f8-419c-af3a-54ef86843ea6 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:44.283152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-349a40a5-6a11-438f-8ed3-a26502932f2a None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:44.284505 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-349a40a5-6a11-438f-8ed3-a26502932f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:44.305569 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:44.317539 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84c30c-629e-4aa4-a788-62b5c202fb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:44.318229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84c30c-629e-4aa4-a788-62b5c202fb03 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:44.318488 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84c30c-629e-4aa4-a788-62b5c202fb03 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:44.318762 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84c30c-629e-4aa4-a788-62b5c202fb03 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:44.319505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84c30c-629e-4aa4-a788-62b5c202fb03 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:44.319840 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84c30c-629e-4aa4-a788-62b5c202fb03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:44.321270 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84c30c-629e-4aa4-a788-62b5c202fb03 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:44.329599 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 57/119] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:47:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:44.334693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84c30c-629e-4aa4-a788-62b5c202fb03 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:44.360947 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 63/120] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:44.371268 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a40a5-6a11-438f-8ed3-a26502932f2a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:44.450461 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84c30c-629e-4aa4-a788-62b5c202fb03 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:44.472276 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a40a5-6a11-438f-8ed3-a26502932f2a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:44.487765 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09c89d-6e50-4175-9566-8cd56342406f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:44.488805 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09c89d-6e50-4175-9566-8cd56342406f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:44.489090 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09c89d-6e50-4175-9566-8cd56342406f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:44.489387 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09c89d-6e50-4175-9566-8cd56342406f None admin] PATH_INFO: `/v3/projects/4c94dfce039f4ff29bec2f9831ec1748` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:44.490581 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09c89d-6e50-4175-9566-8cd56342406f None admin] RBAC: Authorizing `identity:get_project(project_id=4c94dfce039f4ff29bec2f9831ec1748)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:44.492081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09c89d-6e50-4175-9566-8cd56342406f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:44.494044 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 58/121] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:47:44 2022] GET /identity/v3/projects/4c94dfce039f4ff29bec2f9831ec1748 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:44.494641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09c89d-6e50-4175-9566-8cd56342406f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:44.511022 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:44.516958 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0292df2-73fd-49f4-87ef-8cd35c0436c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:44.517679 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0292df2-73fd-49f4-87ef-8cd35c0436c3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:44.517916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0292df2-73fd-49f4-87ef-8cd35c0436c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:44.518196 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0292df2-73fd-49f4-87ef-8cd35c0436c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:44.519074 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0292df2-73fd-49f4-87ef-8cd35c0436c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:44.519653 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0292df2-73fd-49f4-87ef-8cd35c0436c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:44.520950 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0292df2-73fd-49f4-87ef-8cd35c0436c3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:44.531180 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 64/122] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:47:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:44.532515 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0292df2-73fd-49f4-87ef-8cd35c0436c3 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:44.551086 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:44.602155 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09c89d-6e50-4175-9566-8cd56342406f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:44.637654 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552433a-e106-4131-9069-0741d5304980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:44.638876 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552433a-e106-4131-9069-0741d5304980 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:44.639122 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552433a-e106-4131-9069-0741d5304980 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:44.639399 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552433a-e106-4131-9069-0741d5304980 None admin] PATH_INFO: `/v3/users/fa6694f71dec4568a4b7724c3570eff4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:44.640728 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552433a-e106-4131-9069-0741d5304980 None admin] RBAC: Authorizing `identity:get_user(user_id=fa6694f71dec4568a4b7724c3570eff4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:44.642211 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552433a-e106-4131-9069-0741d5304980 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:44.644331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 59/123] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:47:44 2022] GET /identity/v3/users/fa6694f71dec4568a4b7724c3570eff4 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:44.644641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552433a-e106-4131-9069-0741d5304980 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:44.650058 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0292df2-73fd-49f4-87ef-8cd35c0436c3 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:44.684905 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd1ade-1f76-48e4-86c2-2c41dad324b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:44.685803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd1ade-1f76-48e4-86c2-2c41dad324b2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:44.686092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd1ade-1f76-48e4-86c2-2c41dad324b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:44.686371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd1ade-1f76-48e4-86c2-2c41dad324b2 None admin] PATH_INFO: `/v3/users/5060168fc25744178d345904afb6c333` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:44.687569 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd1ade-1f76-48e4-86c2-2c41dad324b2 None admin] RBAC: Authorizing `identity:get_user(user_id=5060168fc25744178d345904afb6c333)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:44.689332 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd1ade-1f76-48e4-86c2-2c41dad324b2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:44.691115 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 65/124] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:47:44 2022] GET /identity/v3/users/5060168fc25744178d345904afb6c333 => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:44.693451 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd1ade-1f76-48e4-86c2-2c41dad324b2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:44.754243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552433a-e106-4131-9069-0741d5304980 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:44.756801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:44.776064 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:44.790546 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d347f-f1e1-433c-95a7-1d974362b424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:44.791344 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d347f-f1e1-433c-95a7-1d974362b424 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:44.791630 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d347f-f1e1-433c-95a7-1d974362b424 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:44.791901 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d347f-f1e1-433c-95a7-1d974362b424 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:44.800961 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd1ade-1f76-48e4-86c2-2c41dad324b2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:44.802252 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d347f-f1e1-433c-95a7-1d974362b424 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:47:44.810747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-fe9d347f-f1e1-433c-95a7-1d974362b424 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:47:44.810747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:47:44.810747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:47:44.810747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:47:44.810747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:47:44.810747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:47:44.810747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:47:44.810747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:47:44.810747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:47:44.810747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:47:44.810747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:47:44.810747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:47:44.810747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:47:44.810747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:47:44.810747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:47:44.810747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:47:44.811610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:47:44.811610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:47:44.811610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:47:44.811610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:47:44.811610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:47:44.811610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:47:44.811610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:47:44.811610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:47:44.811610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:47:44.811610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:47:44.811610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:47:44.811610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:47:44.811610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:47:44.811610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:47:44.811610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:47:44.811610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:47:44.812404 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:47:44.812404 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:47:44.812404 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:47:44.812404 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:47:44.812404 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:47:44.812404 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:47:44.812404 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:47:44.812404 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:47:44.812404 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:47:44.812404 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:47:44.812404 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:47:44.812404 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:47:44.812404 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 60/125] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:47:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:47:44.817131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d347f-f1e1-433c-95a7-1d974362b424 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:44.839838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054c4b2-c551-4baa-b099-2c343d8d355d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:44.840712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054c4b2-c551-4baa-b099-2c343d8d355d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:44.841190 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054c4b2-c551-4baa-b099-2c343d8d355d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:44.841611 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054c4b2-c551-4baa-b099-2c343d8d355d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:44.842624 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f054c4b2-c551-4baa-b099-2c343d8d355d None admin] RBAC: Adding query filter params (role.id=43f9a6d53a0b43c6ada3d7d2eea1c7d2, scope.project.id=4c94dfce039f4ff29bec2f9831ec1748, user.id=f16b644eff4d40d092a047b8dc388112) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:44.843104 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f054c4b2-c551-4baa-b099-2c343d8d355d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:44.844580 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f054c4b2-c551-4baa-b099-2c343d8d355d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:44.860327 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 66/126] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:47:44 2022] GET /identity/v3/role_assignments?user.id=f16b644eff4d40d092a047b8dc388112&scope.project.id=4c94dfce039f4ff29bec2f9831ec1748&role.id=43f9a6d53a0b43c6ada3d7d2eea1c7d2 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:44.860327 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054c4b2-c551-4baa-b099-2c343d8d355d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:44.949015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d347f-f1e1-433c-95a7-1d974362b424 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:44.977465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054c4b2-c551-4baa-b099-2c343d8d355d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:44.998867 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede83fe-b008-4525-9ee2-7ba30a53430c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:44.999877 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede83fe-b008-4525-9ee2-7ba30a53430c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:45.000230 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede83fe-b008-4525-9ee2-7ba30a53430c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:45.000598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede83fe-b008-4525-9ee2-7ba30a53430c None admin] PATH_INFO: `/v3/users/fa6694f71dec4568a4b7724c3570eff4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:45.002734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede83fe-b008-4525-9ee2-7ba30a53430c None admin] RBAC: Authorizing `identity:get_user(user_id=fa6694f71dec4568a4b7724c3570eff4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:45.004795 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede83fe-b008-4525-9ee2-7ba30a53430c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:45.006504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 61/127] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:47:44 2022] GET /identity/v3/users/fa6694f71dec4568a4b7724c3570eff4 => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:45.007889 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede83fe-b008-4525-9ee2-7ba30a53430c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:45.009788 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:45.021870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f475f8ee-0d78-4a87-8620-51c637a24d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:45.022781 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475f8ee-0d78-4a87-8620-51c637a24d1b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:45.023092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475f8ee-0d78-4a87-8620-51c637a24d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:45.023772 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475f8ee-0d78-4a87-8620-51c637a24d1b None admin] PATH_INFO: `/v3/projects/4c94dfce039f4ff29bec2f9831ec1748` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:45.024900 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f475f8ee-0d78-4a87-8620-51c637a24d1b None admin] RBAC: Authorizing `identity:get_project(project_id=4c94dfce039f4ff29bec2f9831ec1748)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:45.026410 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f475f8ee-0d78-4a87-8620-51c637a24d1b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:45.028142 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 67/128] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:47:44 2022] GET /identity/v3/projects/4c94dfce039f4ff29bec2f9831ec1748 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:45.029599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f475f8ee-0d78-4a87-8620-51c637a24d1b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:45.081118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:45.140433 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f475f8ee-0d78-4a87-8620-51c637a24d1b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:45.155004 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede83fe-b008-4525-9ee2-7ba30a53430c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:45.175522 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e79ce7-6cf6-452b-aadb-7591df39e205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:45.176376 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e79ce7-6cf6-452b-aadb-7591df39e205 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:45.176764 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e79ce7-6cf6-452b-aadb-7591df39e205 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:45.177240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e79ce7-6cf6-452b-aadb-7591df39e205 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:45.178124 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e79ce7-6cf6-452b-aadb-7591df39e205 None admin] RBAC: Adding query filter params (name=member) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:45.178536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e79ce7-6cf6-452b-aadb-7591df39e205 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:45.180599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e79ce7-6cf6-452b-aadb-7591df39e205 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:45.189585 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 68/129] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jun 23 19:47:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:45.190504 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e79ce7-6cf6-452b-aadb-7591df39e205 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:45.199046 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f0d6d-0309-4a2e-b81b-5244a8e71179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:45.200179 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f0d6d-0309-4a2e-b81b-5244a8e71179 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:45.201272 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f0d6d-0309-4a2e-b81b-5244a8e71179 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:45.201799 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67f0d6d-0309-4a2e-b81b-5244a8e71179 None admin] PATH_INFO: `/v3/projects/4c94dfce039f4ff29bec2f9831ec1748` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:45.203734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f0d6d-0309-4a2e-b81b-5244a8e71179 None admin] RBAC: Authorizing `identity:get_project(project_id=4c94dfce039f4ff29bec2f9831ec1748)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:45.206207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67f0d6d-0309-4a2e-b81b-5244a8e71179 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:45.209230 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 62/130] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:47:45 2022] GET /identity/v3/projects/4c94dfce039f4ff29bec2f9831ec1748 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:45.210135 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f0d6d-0309-4a2e-b81b-5244a8e71179 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:45.281674 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:45.323333 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67f0d6d-0309-4a2e-b81b-5244a8e71179 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:45.341197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:45.344831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e79ce7-6cf6-452b-aadb-7591df39e205 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:45.366461 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cfd230-abe3-417b-aad7-5a71242abdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:45.367488 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfd230-abe3-417b-aad7-5a71242abdb2 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:45.367863 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfd230-abe3-417b-aad7-5a71242abdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:45.368734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfd230-abe3-417b-aad7-5a71242abdb2 None admin] PATH_INFO: `/v3/projects/4c94dfce039f4ff29bec2f9831ec1748/users/fa6694f71dec4568a4b7724c3570eff4/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:45.371422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cfd230-abe3-417b-aad7-5a71242abdb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c94dfce039f4ff29bec2f9831ec1748, user_id=fa6694f71dec4568a4b7724c3570eff4, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:45.373339 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cfd230-abe3-417b-aad7-5a71242abdb2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:45.387943 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4338edb6-2679-44c0-bbf7-60ff0e5c89ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:45.396603 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338edb6-2679-44c0-bbf7-60ff0e5c89ca None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:45.396603 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338edb6-2679-44c0-bbf7-60ff0e5c89ca None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:45.396603 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338edb6-2679-44c0-bbf7-60ff0e5c89ca None admin] PATH_INFO: `/v3/projects/4c94dfce039f4ff29bec2f9831ec1748` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:45.396603 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4338edb6-2679-44c0-bbf7-60ff0e5c89ca None admin] RBAC: Authorizing `identity:get_project(project_id=4c94dfce039f4ff29bec2f9831ec1748)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:45.396603 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4338edb6-2679-44c0-bbf7-60ff0e5c89ca None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:45.396603 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 69/131] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:47:45 2022] GET /identity/v3/projects/4c94dfce039f4ff29bec2f9831ec1748 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:45.397054 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4338edb6-2679-44c0-bbf7-60ff0e5c89ca None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:45.489721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 63/132] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jun 23 19:47:45 2022] PUT /identity/v3/projects/4c94dfce039f4ff29bec2f9831ec1748/users/fa6694f71dec4568a4b7724c3570eff4/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:47:45.490899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cfd230-abe3-417b-aad7-5a71242abdb2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:45.529016 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4338edb6-2679-44c0-bbf7-60ff0e5c89ca None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:45.556715 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:45.576516 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e22c7d-a4e5-45fc-bed5-3330f22deb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:45.577525 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e22c7d-a4e5-45fc-bed5-3330f22deb59 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:45.580566 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e22c7d-a4e5-45fc-bed5-3330f22deb59 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:45.580566 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e22c7d-a4e5-45fc-bed5-3330f22deb59 None admin] PATH_INFO: `/v3/users/fa6694f71dec4568a4b7724c3570eff4` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:45.580566 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e22c7d-a4e5-45fc-bed5-3330f22deb59 None admin] RBAC: Authorizing `identity:get_user(user_id=fa6694f71dec4568a4b7724c3570eff4)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:45.581637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e22c7d-a4e5-45fc-bed5-3330f22deb59 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:45.583454 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 70/133] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:47:45 2022] GET /identity/v3/users/fa6694f71dec4568a4b7724c3570eff4 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:45.584293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e22c7d-a4e5-45fc-bed5-3330f22deb59 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:45.646284 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cfd230-abe3-417b-aad7-5a71242abdb2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:45.690204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70782c8f-4817-4678-9127-d9e3e575edcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:45.691419 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70782c8f-4817-4678-9127-d9e3e575edcc None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:45.691896 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70782c8f-4817-4678-9127-d9e3e575edcc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:45.692250 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70782c8f-4817-4678-9127-d9e3e575edcc None admin] PATH_INFO: `/v3/projects/4c94dfce039f4ff29bec2f9831ec1748/users/5060168fc25744178d345904afb6c333/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:45.695441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70782c8f-4817-4678-9127-d9e3e575edcc None admin] RBAC: Authorizing `identity:create_grant(project_id=4c94dfce039f4ff29bec2f9831ec1748, user_id=5060168fc25744178d345904afb6c333, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:45.697899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70782c8f-4817-4678-9127-d9e3e575edcc None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:45.698095 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e22c7d-a4e5-45fc-bed5-3330f22deb59 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:45.734988 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74343e5d-e102-47cd-b4f7-f98581c6fa25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:45.735879 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74343e5d-e102-47cd-b4f7-f98581c6fa25 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:45.737240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74343e5d-e102-47cd-b4f7-f98581c6fa25 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:45.737240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74343e5d-e102-47cd-b4f7-f98581c6fa25 None admin] PATH_INFO: `/v3/projects/4c94dfce039f4ff29bec2f9831ec1748/users/fa6694f71dec4568a4b7724c3570eff4/roles/03e4cfc753754643ae08fa909647cfe3` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:45.739282 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74343e5d-e102-47cd-b4f7-f98581c6fa25 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c94dfce039f4ff29bec2f9831ec1748, user_id=fa6694f71dec4568a4b7724c3570eff4, role_id=03e4cfc753754643ae08fa909647cfe3)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:45.740923 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74343e5d-e102-47cd-b4f7-f98581c6fa25 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:45.817776 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 64/134] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jun 23 19:47:45 2022] PUT /identity/v3/projects/4c94dfce039f4ff29bec2f9831ec1748/users/5060168fc25744178d345904afb6c333/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:47:45.819398 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd89df0-e12d-440e-bfc2-b711627d5114 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:45.824269 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd89df0-e12d-440e-bfc2-b711627d5114 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:45.824269 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd89df0-e12d-440e-bfc2-b711627d5114 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:45.824269 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 65/135] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:45.824269 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd89df0-e12d-440e-bfc2-b711627d5114 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:45.864571 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 71/136] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jun 23 19:47:45 2022] PUT /identity/v3/projects/4c94dfce039f4ff29bec2f9831ec1748/users/fa6694f71dec4568a4b7724c3570eff4/roles/03e4cfc753754643ae08fa909647cfe3 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:47:45.867033 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb0cb8-459a-4e13-af36-6cc87a4ad051 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:45.867033 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb0cb8-459a-4e13-af36-6cc87a4ad051 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:45.867033 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb0cb8-459a-4e13-af36-6cc87a4ad051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:45.875493 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:45.901182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b9bb0cb8-459a-4e13-af36-6cc87a4ad051 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:45.902770 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b9bb0cb8-459a-4e13-af36-6cc87a4ad051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:45.992441 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:46.044002 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 72/137] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:46.053983 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd89df0-e12d-440e-bfc2-b711627d5114 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:46.087386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb0cb8-459a-4e13-af36-6cc87a4ad051 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:46.118042 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56f969-2dbd-4349-bc36-52cb8f13c199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:46.119321 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56f969-2dbd-4349-bc36-52cb8f13c199 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:46.119826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56f969-2dbd-4349-bc36-52cb8f13c199 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:46.120435 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56f969-2dbd-4349-bc36-52cb8f13c199 None admin] PATH_INFO: `/v3/projects/5cfe3739530c48c29d910951147203ea` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:46.121771 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd56f969-2dbd-4349-bc36-52cb8f13c199 None admin] RBAC: Authorizing `identity:get_project(project_id=5cfe3739530c48c29d910951147203ea)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:46.123398 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd56f969-2dbd-4349-bc36-52cb8f13c199 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:46.125712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 66/138] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:47:45 2022] GET /identity/v3/projects/5cfe3739530c48c29d910951147203ea => generated 312 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:46.133443 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56f969-2dbd-4349-bc36-52cb8f13c199 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:46.285703 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56f969-2dbd-4349-bc36-52cb8f13c199 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:46.291851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb0cb8-459a-4e13-af36-6cc87a4ad051 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:46.292745 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:46.354002 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3b116-8046-4cac-893f-8b6774914ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:46.354998 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3b116-8046-4cac-893f-8b6774914ead None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:46.355297 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3b116-8046-4cac-893f-8b6774914ead None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:46.355574 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3b116-8046-4cac-893f-8b6774914ead None admin] PATH_INFO: `/v3/projects/5cfe3739530c48c29d910951147203ea/users/fa6694f71dec4568a4b7724c3570eff4/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:46.357688 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d3b116-8046-4cac-893f-8b6774914ead None admin] RBAC: Authorizing `identity:create_grant(project_id=5cfe3739530c48c29d910951147203ea, user_id=fa6694f71dec4568a4b7724c3570eff4, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:46.357898 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10af4c0-a261-44dd-aa7e-897929c5f0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:46.358914 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10af4c0-a261-44dd-aa7e-897929c5f0dc None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:46.359156 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d3b116-8046-4cac-893f-8b6774914ead None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:46.359610 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10af4c0-a261-44dd-aa7e-897929c5f0dc None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:46.360088 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10af4c0-a261-44dd-aa7e-897929c5f0dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:46.365042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10af4c0-a261-44dd-aa7e-897929c5f0dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:46.367388 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10af4c0-a261-44dd-aa7e-897929c5f0dc None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:46.369065 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 73/139] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:47:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:46.374901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10af4c0-a261-44dd-aa7e-897929c5f0dc None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:46.425784 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:46.463708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 67/140] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jun 23 19:47:46 2022] PUT /identity/v3/projects/5cfe3739530c48c29d910951147203ea/users/fa6694f71dec4568a4b7724c3570eff4/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:47:46.486325 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10af4c0-a261-44dd-aa7e-897929c5f0dc None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:46.524625 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d72f6-4271-47ad-affd-ef7dd5d6ab45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:46.525313 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d72f6-4271-47ad-affd-ef7dd5d6ab45 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:46.525603 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d72f6-4271-47ad-affd-ef7dd5d6ab45 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:46.525881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d72f6-4271-47ad-affd-ef7dd5d6ab45 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:46.526613 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d72f6-4271-47ad-affd-ef7dd5d6ab45 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:46.528435 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1d72f6-4271-47ad-affd-ef7dd5d6ab45 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:46.645167 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 74/141] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jun 23 19:47:46 2022] POST /identity/v3/users => generated 283 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 19:47:47.321999 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c4297-3aba-4a18-a602-e2116673671e None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:47.322526 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c4297-3aba-4a18-a602-e2116673671e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:47.322621 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c4297-3aba-4a18-a602-e2116673671e None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:47.323924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 68/142] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:47.330228 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eade87-e99d-4362-adb2-6e2ea1760ba7 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:47.330520 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eade87-e99d-4362-adb2-6e2ea1760ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:47.330857 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eade87-e99d-4362-adb2-6e2ea1760ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:47.352912 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-24eade87-e99d-4362-adb2-6e2ea1760ba7 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:47.354110 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-24eade87-e99d-4362-adb2-6e2ea1760ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:47.434156 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 75/143] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:47.458120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c4297-3aba-4a18-a602-e2116673671e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:47.587011 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c4297-3aba-4a18-a602-e2116673671e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:47.627334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ee648-c600-4293-a23a-ae4442c303b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:47.628141 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ee648-c600-4293-a23a-ae4442c303b5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:47.628451 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ee648-c600-4293-a23a-ae4442c303b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:47.628727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ee648-c600-4293-a23a-ae4442c303b5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:47.642094 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0ee648-c600-4293-a23a-ae4442c303b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:47:47.650687 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-7e0ee648-c600-4293-a23a-ae4442c303b5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 19:47:47.650687 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:47:47.650687 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:47:47.650687 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:47:47.650687 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:47:47.650687 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:47:47.650687 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:47:47.650687 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:47:47.650687 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:47:47.650687 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:47:47.650687 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:47:47.650687 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:47:47.650687 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:47:47.650687 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:47:47.650687 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:47:47.650687 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:47:47.651520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:47:47.651520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:47:47.651520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:47:47.651520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:47:47.651520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:47:47.651520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:47:47.651520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:47:47.651520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:47:47.651520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:47:47.651520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:47:47.651520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:47:47.651520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:47:47.651520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:47:47.651520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:47:47.651520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:47:47.651520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:47:47.652258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:47:47.652258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:47:47.652258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:47:47.652258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:47:47.652258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:47:47.652258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:47:47.652258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:47:47.652258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:47:47.652258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:47:47.652258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:47:47.652258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 19:47:47.652258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:47:47.653317 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 69/144] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:47:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:47:47.658213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eade87-e99d-4362-adb2-6e2ea1760ba7 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:47.697465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:47.769493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eade87-e99d-4362-adb2-6e2ea1760ba7 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:47.809167 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae14c6-1ccc-4489-b9ad-c7b49e43b37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:47.809837 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae14c6-1ccc-4489-b9ad-c7b49e43b37d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:47.810118 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae14c6-1ccc-4489-b9ad-c7b49e43b37d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:47.810391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae14c6-1ccc-4489-b9ad-c7b49e43b37d None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:47.811142 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae14c6-1ccc-4489-b9ad-c7b49e43b37d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:47.811445 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae14c6-1ccc-4489-b9ad-c7b49e43b37d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:47.813251 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae14c6-1ccc-4489-b9ad-c7b49e43b37d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:47.826918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 76/145] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jun 23 19:47:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:47.828378 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ee648-c600-4293-a23a-ae4442c303b5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:47.856194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:47.933851 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ee648-c600-4293-a23a-ae4442c303b5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:47.971738 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e95637e-01bb-4887-92aa-5e49a629d57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:47.972643 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95637e-01bb-4887-92aa-5e49a629d57b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:47.972925 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95637e-01bb-4887-92aa-5e49a629d57b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:47.973195 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95637e-01bb-4887-92aa-5e49a629d57b None admin] PATH_INFO: `/v3/users/f16b644eff4d40d092a047b8dc388112` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:47.974544 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e95637e-01bb-4887-92aa-5e49a629d57b None admin] RBAC: Authorizing `identity:get_user(user_id=f16b644eff4d40d092a047b8dc388112)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:47.976036 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e95637e-01bb-4887-92aa-5e49a629d57b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:47.977874 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 70/146] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:47:47 2022] GET /identity/v3/users/f16b644eff4d40d092a047b8dc388112 => generated 289 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:47.984521 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae14c6-1ccc-4489-b9ad-c7b49e43b37d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:48.003501 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7d6f7-7495-4d73-9383-1091d7c6c633 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:48.003792 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7d6f7-7495-4d73-9383-1091d7c6c633 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:48.004113 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7d6f7-7495-4d73-9383-1091d7c6c633 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:48.005683 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 71/147] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:48.011382 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6105b596-2696-4fee-8a2e-131499fff5a0 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:48.011668 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6105b596-2696-4fee-8a2e-131499fff5a0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:48.011947 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6105b596-2696-4fee-8a2e-131499fff5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:48.041406 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6105b596-2696-4fee-8a2e-131499fff5a0 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:48.045264 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6105b596-2696-4fee-8a2e-131499fff5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:48.182946 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 72/148] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:48.184424 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:48.198433 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae14c6-1ccc-4489-b9ad-c7b49e43b37d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:48.221598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6105b596-2696-4fee-8a2e-131499fff5a0 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:48.263287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:48.285060 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50415c-194f-4d78-8910-d5d5cf74f982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:48.285945 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50415c-194f-4d78-8910-d5d5cf74f982 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:48.286232 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50415c-194f-4d78-8910-d5d5cf74f982 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:48.286512 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50415c-194f-4d78-8910-d5d5cf74f982 None admin] PATH_INFO: `/v3/projects/4c94dfce039f4ff29bec2f9831ec1748` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:48.287770 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50415c-194f-4d78-8910-d5d5cf74f982 None admin] RBAC: Authorizing `identity:get_project(project_id=4c94dfce039f4ff29bec2f9831ec1748)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:48.289820 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50415c-194f-4d78-8910-d5d5cf74f982 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:48.291773 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 77/149] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:47:47 2022] GET /identity/v3/projects/4c94dfce039f4ff29bec2f9831ec1748 => generated 298 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:48.322511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50415c-194f-4d78-8910-d5d5cf74f982 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:48.368507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6105b596-2696-4fee-8a2e-131499fff5a0 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:48.418400 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740806b3-2ea0-4d2a-b4b7-027550eed4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:48.419398 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740806b3-2ea0-4d2a-b4b7-027550eed4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:48.419683 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740806b3-2ea0-4d2a-b4b7-027550eed4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:48.419965 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740806b3-2ea0-4d2a-b4b7-027550eed4b3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:48.433590 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740806b3-2ea0-4d2a-b4b7-027550eed4b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:47:48.446176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-740806b3-2ea0-4d2a-b4b7-027550eed4b3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:47:48.446176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:47:48.446176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:47:48.446176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:47:48.446176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:47:48.446176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:47:48.446176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:47:48.446176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:47:48.446176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:47:48.446176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:47:48.446176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:47:48.446176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:47:48.446176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:47:48.446176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:47:48.446176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:47:48.446176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:47:48.447055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:47:48.447055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:47:48.447055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:47:48.447055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:47:48.447055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:47:48.447055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:47:48.447055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:47:48.447055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:47:48.447055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:47:48.447055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:47:48.447055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:47:48.447055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:47:48.447055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:47:48.447055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:47:48.447055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:47:48.447055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:47:48.447799 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:47:48.447799 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:47:48.447799 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:47:48.447799 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:47:48.447799 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:47:48.447799 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:47:48.447799 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:47:48.447799 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:47:48.447799 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:47:48.447799 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:47:48.447799 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:47:48.447799 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:47:48.447799 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 73/150] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:47:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:47:48.454683 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740806b3-2ea0-4d2a-b4b7-027550eed4b3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:48.588656 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:48.631775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50415c-194f-4d78-8910-d5d5cf74f982 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:48.638451 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740806b3-2ea0-4d2a-b4b7-027550eed4b3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:48.692934 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ce5a1-ce15-4d46-9226-b8da5c3ee9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:48.694019 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ce5a1-ce15-4d46-9226-b8da5c3ee9a2 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:48.694397 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ce5a1-ce15-4d46-9226-b8da5c3ee9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:48.694822 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ce5a1-ce15-4d46-9226-b8da5c3ee9a2 None admin] PATH_INFO: `/v3/projects/4c94dfce039f4ff29bec2f9831ec1748/users/f16b644eff4d40d092a047b8dc388112/roles/43f9a6d53a0b43c6ada3d7d2eea1c7d2` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:48.697059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52ce5a1-ce15-4d46-9226-b8da5c3ee9a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=4c94dfce039f4ff29bec2f9831ec1748, user_id=f16b644eff4d40d092a047b8dc388112, role_id=43f9a6d53a0b43c6ada3d7d2eea1c7d2)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:48.698748 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52ce5a1-ce15-4d46-9226-b8da5c3ee9a2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:48.722880 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db579052-32ff-4076-9e01-b2d7dadee434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:48.723790 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db579052-32ff-4076-9e01-b2d7dadee434 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:48.724219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db579052-32ff-4076-9e01-b2d7dadee434 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:48.736363 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db579052-32ff-4076-9e01-b2d7dadee434 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:48.736363 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db579052-32ff-4076-9e01-b2d7dadee434 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:48.736363 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db579052-32ff-4076-9e01-b2d7dadee434 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:48.736363 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db579052-32ff-4076-9e01-b2d7dadee434 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:48.744233 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 74/151] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jun 23 19:47:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:47:48.749580 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db579052-32ff-4076-9e01-b2d7dadee434 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:48.826659 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 78/152] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jun 23 19:47:48 2022] PUT /identity/v3/projects/4c94dfce039f4ff29bec2f9831ec1748/users/f16b644eff4d40d092a047b8dc388112/roles/43f9a6d53a0b43c6ada3d7d2eea1c7d2 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 19:47:48.878966 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db579052-32ff-4076-9e01-b2d7dadee434 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:48.920048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fddf658-53b1-4ebc-a958-a3274b3963ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:48.920977 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddf658-53b1-4ebc-a958-a3274b3963ee None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:48.921258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddf658-53b1-4ebc-a958-a3274b3963ee None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:48.921526 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddf658-53b1-4ebc-a958-a3274b3963ee None admin] PATH_INFO: `/v3/users/fa6694f71dec4568a4b7724c3570eff4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:48.922775 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fddf658-53b1-4ebc-a958-a3274b3963ee None admin] RBAC: Authorizing `identity:get_user(user_id=fa6694f71dec4568a4b7724c3570eff4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:48.924236 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fddf658-53b1-4ebc-a958-a3274b3963ee None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:48.926081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 75/153] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:47:48 2022] GET /identity/v3/users/fa6694f71dec4568a4b7724c3570eff4 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:47:48.931878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ce5a1-ce15-4d46-9226-b8da5c3ee9a2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:48.971473 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:49.068114 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ce5a1-ce15-4d46-9226-b8da5c3ee9a2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:49.104936 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbad54-fe06-40b4-b5f5-47ef045b3b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:49.105934 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbad54-fe06-40b4-b5f5-47ef045b3b1c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:49.106408 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbad54-fe06-40b4-b5f5-47ef045b3b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:49.106900 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbad54-fe06-40b4-b5f5-47ef045b3b1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:49.139311 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408583a-9432-48ba-b0c1-d8b6be0e4a79 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:49.139636 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408583a-9432-48ba-b0c1-d8b6be0e4a79 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:49.139877 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408583a-9432-48ba-b0c1-d8b6be0e4a79 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:49.141301 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 76/154] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:47:49.161822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded1cf8-1294-48ca-b0a1-49bf84ac07fe None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:49.162146 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded1cf8-1294-48ca-b0a1-49bf84ac07fe None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:49.162384 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded1cf8-1294-48ca-b0a1-49bf84ac07fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:49.194450 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebbad54-fe06-40b4-b5f5-47ef045b3b1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:49.196327 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebbad54-fe06-40b4-b5f5-47ef045b3b1c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:49.209357 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-bded1cf8-1294-48ca-b0a1-49bf84ac07fe None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:49.210629 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-bded1cf8-1294-48ca-b0a1-49bf84ac07fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:49.224223 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:49.227702 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:49.312376 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 79/155] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:47:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:47:49.318270 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbad54-fe06-40b4-b5f5-47ef045b3b1c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:49.319246 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 77/156] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 19:47:49.353291 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded1cf8-1294-48ca-b0a1-49bf84ac07fe None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:49.445215 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbad54-fe06-40b4-b5f5-47ef045b3b1c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:49.479182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:49.494408 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86960ec0-3cfb-466f-84ad-8ecc9e73f348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:49.495273 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86960ec0-3cfb-466f-84ad-8ecc9e73f348 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:49.495658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86960ec0-3cfb-466f-84ad-8ecc9e73f348 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:49.500512 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86960ec0-3cfb-466f-84ad-8ecc9e73f348 None admin] PATH_INFO: `/v3/projects/4c94dfce039f4ff29bec2f9831ec1748` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:49.502005 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86960ec0-3cfb-466f-84ad-8ecc9e73f348 None admin] RBAC: Authorizing `identity:get_project(project_id=4c94dfce039f4ff29bec2f9831ec1748)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:49.503764 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86960ec0-3cfb-466f-84ad-8ecc9e73f348 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:49.505695 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 80/157] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:47:49 2022] GET /identity/v3/projects/4c94dfce039f4ff29bec2f9831ec1748 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:47:49.511634 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86960ec0-3cfb-466f-84ad-8ecc9e73f348 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:49.572953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded1cf8-1294-48ca-b0a1-49bf84ac07fe None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:49.630858 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86960ec0-3cfb-466f-84ad-8ecc9e73f348 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:49.657874 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c27bd38-011c-4e7f-9a4f-a29dd6f3babd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:49.659061 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27bd38-011c-4e7f-9a4f-a29dd6f3babd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:49.659536 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27bd38-011c-4e7f-9a4f-a29dd6f3babd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:49.660028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27bd38-011c-4e7f-9a4f-a29dd6f3babd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:49.671802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a3ae4-2096-4f1e-ba0b-b04db7fb21be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:49.672482 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a3ae4-2096-4f1e-ba0b-b04db7fb21be None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:49.672763 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a3ae4-2096-4f1e-ba0b-b04db7fb21be None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:49.673035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a3ae4-2096-4f1e-ba0b-b04db7fb21be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:49.673875 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a3ae4-2096-4f1e-ba0b-b04db7fb21be None admin] RBAC: Adding query filter params (role.id=860491bfbb574772a7a213d6a25962e0, scope.project.id=4c94dfce039f4ff29bec2f9831ec1748, user.id=fa6694f71dec4568a4b7724c3570eff4) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:49.674184 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a3ae4-2096-4f1e-ba0b-b04db7fb21be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:49.675750 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a3ae4-2096-4f1e-ba0b-b04db7fb21be None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:49.683366 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c27bd38-011c-4e7f-9a4f-a29dd6f3babd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:47:49.686710 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 81/158] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:47:49 2022] GET /identity/v3/role_assignments?user.id=fa6694f71dec4568a4b7724c3570eff4&scope.project.id=4c94dfce039f4ff29bec2f9831ec1748&role.id=860491bfbb574772a7a213d6a25962e0 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:47:49.696918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-4c27bd38-011c-4e7f-9a4f-a29dd6f3babd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 19:47:49.696918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:47:49.696918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:47:49.696918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:47:49.696918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:47:49.696918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:47:49.696918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:47:49.696918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:47:49.696918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:47:49.696918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:47:49.696918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:47:49.696918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:47:49.696918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:47:49.696918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:47:49.696918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:47:49.696918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:47:49.697826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:47:49.697826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:47:49.697826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:47:49.697826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:47:49.697826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:47:49.697826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:47:49.697826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:47:49.697826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:47:49.697826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:47:49.697826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:47:49.697826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:47:49.697826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:47:49.697826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:47:49.697826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:47:49.697826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:47:49.697826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:47:49.698665 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:47:49.698665 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:47:49.698665 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:47:49.698665 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:47:49.698665 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:47:49.698665 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:47:49.698665 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:47:49.698665 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:47:49.698665 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:47:49.698665 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:47:49.698665 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 19:47:49.698665 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:47:49.701049 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 78/159] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jun 23 19:47:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:47:49.707009 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a3ae4-2096-4f1e-ba0b-b04db7fb21be None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:49.720992 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436463a2-27c7-43d6-b9c4-82438dcd90db None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:49.721611 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436463a2-27c7-43d6-b9c4-82438dcd90db None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:49.722150 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436463a2-27c7-43d6-b9c4-82438dcd90db None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:49.723830 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 79/160] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:47:49.736937 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b39989-9bad-4a0b-abc6-4b4089a61da0 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:49.737524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b39989-9bad-4a0b-abc6-4b4089a61da0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:49.737994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b39989-9bad-4a0b-abc6-4b4089a61da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:49.744952 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:49.760896 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-29b39989-9bad-4a0b-abc6-4b4089a61da0 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:49.762482 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-29b39989-9bad-4a0b-abc6-4b4089a61da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:49.776835 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:49.830425 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a3ae4-2096-4f1e-ba0b-b04db7fb21be None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:49.873135 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d5a4e-11a8-4c15-a3d0-3ecbb580dc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:49.874819 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d5a4e-11a8-4c15-a3d0-3ecbb580dc7e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:49.875097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d5a4e-11a8-4c15-a3d0-3ecbb580dc7e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:49.875369 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d5a4e-11a8-4c15-a3d0-3ecbb580dc7e None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:49.876130 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d5a4e-11a8-4c15-a3d0-3ecbb580dc7e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:49.876456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d5a4e-11a8-4c15-a3d0-3ecbb580dc7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:49.877866 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d5a4e-11a8-4c15-a3d0-3ecbb580dc7e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:49.901845 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 82/161] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:47:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:49.905954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d5a4e-11a8-4c15-a3d0-3ecbb580dc7e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:49.923710 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 80/162] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 19:47:49.971560 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b39989-9bad-4a0b-abc6-4b4089a61da0 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:50.084110 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d5a4e-11a8-4c15-a3d0-3ecbb580dc7e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:50.121726 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:50.154110 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b39989-9bad-4a0b-abc6-4b4089a61da0 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:50.169012 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5404ed92-1fbd-463f-89d0-80e0370026a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:50.169899 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404ed92-1fbd-463f-89d0-80e0370026a4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:50.170180 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404ed92-1fbd-463f-89d0-80e0370026a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:50.170452 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404ed92-1fbd-463f-89d0-80e0370026a4 None admin] PATH_INFO: `/v3/users/fa6694f71dec4568a4b7724c3570eff4` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:50.171726 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5404ed92-1fbd-463f-89d0-80e0370026a4 None admin] RBAC: Authorizing `identity:get_user(user_id=fa6694f71dec4568a4b7724c3570eff4)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:50.173246 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5404ed92-1fbd-463f-89d0-80e0370026a4 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:50.175079 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 83/163] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:47:49 2022] GET /identity/v3/users/fa6694f71dec4568a4b7724c3570eff4 => generated 275 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:50.181076 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5404ed92-1fbd-463f-89d0-80e0370026a4 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:50.209598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:50.217377 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8870fcf0-4063-4df6-8751-5ef4b8d01164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:50.218189 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870fcf0-4063-4df6-8751-5ef4b8d01164 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:50.218465 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870fcf0-4063-4df6-8751-5ef4b8d01164 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:50.218733 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870fcf0-4063-4df6-8751-5ef4b8d01164 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:50.240340 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8870fcf0-4063-4df6-8751-5ef4b8d01164 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:47:50.248788 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-8870fcf0-4063-4df6-8751-5ef4b8d01164 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:47:50.248788 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:47:50.248788 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:47:50.248788 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:47:50.248788 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:47:50.248788 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:47:50.248788 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:47:50.248788 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:47:50.248788 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:47:50.248788 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:47:50.248788 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:47:50.248788 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:47:50.248788 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:47:50.248788 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:47:50.248788 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:47:50.248788 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:47:50.249737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:47:50.249737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:47:50.249737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:47:50.249737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:47:50.249737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:47:50.249737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:47:50.249737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:47:50.249737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:47:50.249737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:47:50.249737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:47:50.249737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:47:50.249737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:47:50.249737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:47:50.249737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:47:50.249737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:47:50.249737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:47:50.250524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:47:50.250524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:47:50.250524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:47:50.250524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:47:50.250524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:47:50.250524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:47:50.250524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:47:50.250524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:47:50.250524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:47:50.250524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:47:50.250524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:47:50.250524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:47:50.254269 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 81/164] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:47:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:47:50.261462 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8870fcf0-4063-4df6-8751-5ef4b8d01164 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:50.464097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5404ed92-1fbd-463f-89d0-80e0370026a4 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:50.476134 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8870fcf0-4063-4df6-8751-5ef4b8d01164 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:50.553032 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d07fb-9056-4eb6-829d-9a430bc7df63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:50.553717 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d07fb-9056-4eb6-829d-9a430bc7df63 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:50.554555 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d07fb-9056-4eb6-829d-9a430bc7df63 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:50.554837 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d07fb-9056-4eb6-829d-9a430bc7df63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:50.557896 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e8b3dd-7efb-4a41-8deb-58a8a1b3e69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:50.558805 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8b3dd-7efb-4a41-8deb-58a8a1b3e69b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:50.559278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8b3dd-7efb-4a41-8deb-58a8a1b3e69b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:50.559739 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e8b3dd-7efb-4a41-8deb-58a8a1b3e69b None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:50.560709 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e8b3dd-7efb-4a41-8deb-58a8a1b3e69b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:50.561206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e8b3dd-7efb-4a41-8deb-58a8a1b3e69b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:50.563058 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e8b3dd-7efb-4a41-8deb-58a8a1b3e69b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:50.579633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 82/165] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:47:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:47:50.581714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12da27-c23e-4b8b-a5bf-6ad5752ad165 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:50.582193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12da27-c23e-4b8b-a5bf-6ad5752ad165 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:50.582656 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12da27-c23e-4b8b-a5bf-6ad5752ad165 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:50.584114 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 83/166] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:50.589847 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149accdb-ea1d-4ba1-9366-925e45cf2746 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:50.590408 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149accdb-ea1d-4ba1-9366-925e45cf2746 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:50.590877 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149accdb-ea1d-4ba1-9366-925e45cf2746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:50.617180 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-149accdb-ea1d-4ba1-9366-925e45cf2746 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:50.619002 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-149accdb-ea1d-4ba1-9366-925e45cf2746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:50.680710 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:50.696325 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:50.710543 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d07fb-9056-4eb6-829d-9a430bc7df63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:50.712105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d07fb-9056-4eb6-829d-9a430bc7df63 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:50.713709 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 84/167] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 19:47:50.714835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149accdb-ea1d-4ba1-9366-925e45cf2746 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:50.859639 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149accdb-ea1d-4ba1-9366-925e45cf2746 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:50.926801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 84/168] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:47:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 742 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 19:47:50.927646 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d07fb-9056-4eb6-829d-9a430bc7df63 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:50.945521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7d795-40dd-4887-bb4d-0b9eecf3d3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:50.946417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7d795-40dd-4887-bb4d-0b9eecf3d3a7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:50.946700 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7d795-40dd-4887-bb4d-0b9eecf3d3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:50.946976 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7d795-40dd-4887-bb4d-0b9eecf3d3a7 None admin] PATH_INFO: `/v3/users/5060168fc25744178d345904afb6c333` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:50.948198 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7d795-40dd-4887-bb4d-0b9eecf3d3a7 None admin] RBAC: Authorizing `identity:get_user(user_id=5060168fc25744178d345904afb6c333)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:50.949685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7d795-40dd-4887-bb4d-0b9eecf3d3a7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:50.951330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 85/169] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:47:50 2022] GET /identity/v3/users/5060168fc25744178d345904afb6c333 => generated 247 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:50.960689 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7d795-40dd-4887-bb4d-0b9eecf3d3a7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:51.060923 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:51.119135 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:51.144852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7d795-40dd-4887-bb4d-0b9eecf3d3a7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:51.155887 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d07fb-9056-4eb6-829d-9a430bc7df63 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:51.241416 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71036acd-84d3-4108-a8fb-609a5aa69767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:51.241985 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71036acd-84d3-4108-a8fb-609a5aa69767 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:51.243037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71036acd-84d3-4108-a8fb-609a5aa69767 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:51.243037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71036acd-84d3-4108-a8fb-609a5aa69767 None admin] PATH_INFO: `/v3/projects/4c94dfce039f4ff29bec2f9831ec1748` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:51.243233 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24482ecc-8628-4017-8be9-c154724058e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:51.243696 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71036acd-84d3-4108-a8fb-609a5aa69767 None admin] RBAC: Authorizing `identity:get_project(project_id=4c94dfce039f4ff29bec2f9831ec1748)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:51.243909 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24482ecc-8628-4017-8be9-c154724058e5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:51.244197 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24482ecc-8628-4017-8be9-c154724058e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:51.244493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24482ecc-8628-4017-8be9-c154724058e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:51.245186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71036acd-84d3-4108-a8fb-609a5aa69767 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:51.247264 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 86/170] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:47:50 2022] GET /identity/v3/projects/4c94dfce039f4ff29bec2f9831ec1748 => generated 298 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:51.249031 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71036acd-84d3-4108-a8fb-609a5aa69767 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:51.279452 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24482ecc-8628-4017-8be9-c154724058e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:47:51.304480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-24482ecc-8628-4017-8be9-c154724058e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:47:51.304480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:47:51.304480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:47:51.304480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:47:51.304480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:47:51.304480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:47:51.304480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:47:51.304480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:47:51.304480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:47:51.304480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:47:51.304480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:47:51.304480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:47:51.304480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:47:51.304480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:47:51.304480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:47:51.304480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:47:51.305411 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:47:51.305411 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:47:51.305411 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:47:51.305411 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:47:51.305411 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:47:51.305411 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:47:51.305411 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:47:51.305411 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:47:51.305411 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:47:51.305411 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:47:51.305411 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:47:51.305411 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:47:51.305411 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:47:51.305411 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:47:51.305411 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:47:51.305411 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:47:51.306195 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:47:51.306195 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:47:51.306195 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:47:51.306195 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:47:51.306195 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:47:51.306195 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:47:51.306195 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:47:51.306195 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:47:51.306195 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:47:51.306195 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:47:51.306195 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:47:51.306195 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:47:51.308712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 85/171] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:47:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:47:51.309796 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24482ecc-8628-4017-8be9-c154724058e5 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:51.470814 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71036acd-84d3-4108-a8fb-609a5aa69767 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:51.495440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24482ecc-8628-4017-8be9-c154724058e5 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:51.500166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:51.521935 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fecf86-4e42-4262-abd4-013ebe0652ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:51.523572 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fecf86-4e42-4262-abd4-013ebe0652ac None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:51.523863 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fecf86-4e42-4262-abd4-013ebe0652ac None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:51.524143 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fecf86-4e42-4262-abd4-013ebe0652ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:51.537239 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c557529-c903-408a-813f-a820ec752f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:51.538224 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c557529-c903-408a-813f-a820ec752f91 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:51.538703 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c557529-c903-408a-813f-a820ec752f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:51.539171 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c557529-c903-408a-813f-a820ec752f91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:51.540256 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c557529-c903-408a-813f-a820ec752f91 None admin] RBAC: Adding query filter params (role.id=03e4cfc753754643ae08fa909647cfe3, scope.project.id=4c94dfce039f4ff29bec2f9831ec1748, user.id=fa6694f71dec4568a4b7724c3570eff4) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:51.540856 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c557529-c903-408a-813f-a820ec752f91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:51.543603 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c557529-c903-408a-813f-a820ec752f91 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:51.550310 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 86/172] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:47:51 2022] GET /identity/v3/role_assignments?user.id=fa6694f71dec4568a4b7724c3570eff4&scope.project.id=4c94dfce039f4ff29bec2f9831ec1748&role.id=03e4cfc753754643ae08fa909647cfe3 => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:51.558214 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be4002-e803-449c-ba71-7a2c542a4a01 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:51.558760 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be4002-e803-449c-ba71-7a2c542a4a01 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:51.559388 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be4002-e803-449c-ba71-7a2c542a4a01 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:51.561350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 87/173] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:51.567223 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be4002-e803-449c-ba71-7a2c542a4a01 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:51.589123 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:51.604703 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fecf86-4e42-4262-abd4-013ebe0652ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:51.606220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fecf86-4e42-4262-abd4-013ebe0652ac None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:51.733056 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 87/174] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:47:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:47:51.734332 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af0ea5-f8e3-4843-b39a-eaf804603f5d None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:51.734620 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af0ea5-f8e3-4843-b39a-eaf804603f5d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:51.734895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af0ea5-f8e3-4843-b39a-eaf804603f5d None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:51.736670 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 88/175] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:51.737590 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbca559-95b3-4a40-8d6d-a715b17e742d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:51.738579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbca559-95b3-4a40-8d6d-a715b17e742d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:51.738875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbca559-95b3-4a40-8d6d-a715b17e742d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:51.776761 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ffbca559-95b3-4a40-8d6d-a715b17e742d None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:51.778123 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ffbca559-95b3-4a40-8d6d-a715b17e742d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:51.795089 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be4002-e803-449c-ba71-7a2c542a4a01 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:51.813745 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:51.843080 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c79168-a949-4221-8b17-660ede40a98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:51.844077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c79168-a949-4221-8b17-660ede40a98d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:51.844619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c79168-a949-4221-8b17-660ede40a98d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:51.845125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c79168-a949-4221-8b17-660ede40a98d None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:51.846069 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c79168-a949-4221-8b17-660ede40a98d None admin] RBAC: Adding query filter params (name=member) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:51.846565 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c79168-a949-4221-8b17-660ede40a98d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:51.848255 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c79168-a949-4221-8b17-660ede40a98d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:51.858137 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 88/176] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jun 23 19:47:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:51.859344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c79168-a949-4221-8b17-660ede40a98d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:51.879092 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 89/177] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:51.883528 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae410a-3da3-42f8-bfe6-88989c37b9cb None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:51.883528 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae410a-3da3-42f8-bfe6-88989c37b9cb None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:51.883528 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae410a-3da3-42f8-bfe6-88989c37b9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:51.901197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-55ae410a-3da3-42f8-bfe6-88989c37b9cb None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:51.902528 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-55ae410a-3da3-42f8-bfe6-88989c37b9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:51.964349 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:51.989238 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 90/178] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:51.990012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae410a-3da3-42f8-bfe6-88989c37b9cb None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:52.000864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c79168-a949-4221-8b17-660ede40a98d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:52.052278 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea4d1ba-6a69-44a1-9145-c3aac5b34aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:52.053394 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4d1ba-6a69-44a1-9145-c3aac5b34aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:52.053502 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4d1ba-6a69-44a1-9145-c3aac5b34aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:52.053699 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4d1ba-6a69-44a1-9145-c3aac5b34aa8 None admin] PATH_INFO: `/v3/projects/4c94dfce039f4ff29bec2f9831ec1748` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:52.055481 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea4d1ba-6a69-44a1-9145-c3aac5b34aa8 None admin] RBAC: Authorizing `identity:get_project(project_id=4c94dfce039f4ff29bec2f9831ec1748)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:52.057002 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea4d1ba-6a69-44a1-9145-c3aac5b34aa8 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:52.058864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 89/179] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:47:51 2022] GET /identity/v3/projects/4c94dfce039f4ff29bec2f9831ec1748 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:52.059464 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea4d1ba-6a69-44a1-9145-c3aac5b34aa8 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:52.080996 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:52.102473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae410a-3da3-42f8-bfe6-88989c37b9cb None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:52.138595 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b58897-71a2-4716-b248-470bda156518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:52.139466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b58897-71a2-4716-b248-470bda156518 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:52.139744 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b58897-71a2-4716-b248-470bda156518 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:52.140012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b58897-71a2-4716-b248-470bda156518 None admin] PATH_INFO: `/v3/users/fa6694f71dec4568a4b7724c3570eff4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:52.141453 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b58897-71a2-4716-b248-470bda156518 None admin] RBAC: Authorizing `identity:get_user(user_id=fa6694f71dec4568a4b7724c3570eff4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:52.142905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b58897-71a2-4716-b248-470bda156518 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:52.149024 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 91/180] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:47:51 2022] GET /identity/v3/users/fa6694f71dec4568a4b7724c3570eff4 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:52.149024 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7492188-27e4-4b29-abf4-5100a5340455 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:52.149024 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7492188-27e4-4b29-abf4-5100a5340455 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:52.149024 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7492188-27e4-4b29-abf4-5100a5340455 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:52.149024 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 92/181] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:52.149024 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7492188-27e4-4b29-abf4-5100a5340455 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:52.175122 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea4d1ba-6a69-44a1-9145-c3aac5b34aa8 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:52.210974 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573ee80-2e2d-4347-8eaa-aaa93dce075a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:52.211764 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573ee80-2e2d-4347-8eaa-aaa93dce075a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:52.212044 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573ee80-2e2d-4347-8eaa-aaa93dce075a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:52.212332 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573ee80-2e2d-4347-8eaa-aaa93dce075a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:52.220053 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:52.223105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573ee80-2e2d-4347-8eaa-aaa93dce075a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:47:52.237371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-e573ee80-2e2d-4347-8eaa-aaa93dce075a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:47:52.237371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:47:52.237371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:47:52.237371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:47:52.237371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:47:52.237371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:47:52.237371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:47:52.237371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:47:52.237371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:47:52.237371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:47:52.237371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:47:52.237371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:47:52.237371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:47:52.237371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:47:52.237371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:47:52.237371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:47:52.238444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:47:52.238444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:47:52.238444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:47:52.238444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:47:52.238444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:47:52.238444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:47:52.238444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:47:52.238444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:47:52.238444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:47:52.238444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:47:52.238444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:47:52.238444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:47:52.238444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:47:52.238444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:47:52.238444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:47:52.238444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:47:52.239495 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:47:52.239495 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:47:52.239495 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:47:52.239495 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:47:52.239495 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:47:52.239495 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:47:52.239495 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:47:52.239495 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:47:52.239495 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:47:52.239495 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:47:52.239495 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:47:52.239495 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:47:52.239495 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 90/182] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:47:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:47:52.239495 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573ee80-2e2d-4347-8eaa-aaa93dce075a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:52.264472 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7492188-27e4-4b29-abf4-5100a5340455 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:52.300461 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1708f-5534-43c4-8593-f98576d92562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:52.301269 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1708f-5534-43c4-8593-f98576d92562 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:52.301557 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1708f-5534-43c4-8593-f98576d92562 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:52.301831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade1708f-5534-43c4-8593-f98576d92562 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:52.313336 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade1708f-5534-43c4-8593-f98576d92562 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:47:52.321327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-ade1708f-5534-43c4-8593-f98576d92562 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 19:47:52.321327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:47:52.321327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:47:52.321327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:47:52.321327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:47:52.321327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:47:52.321327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:47:52.321327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:47:52.321327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:47:52.321327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:47:52.321327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:47:52.321327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:47:52.321327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:47:52.321327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:47:52.321327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:47:52.321327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:47:52.322193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:47:52.322193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:47:52.322193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:47:52.322193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:47:52.322193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:47:52.322193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:47:52.322193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:47:52.322193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:47:52.322193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:47:52.322193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:47:52.322193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:47:52.322193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:47:52.322193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:47:52.322193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:47:52.322193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:47:52.322193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:47:52.322995 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:47:52.322995 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:47:52.322995 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:47:52.322995 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:47:52.322995 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:47:52.322995 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:47:52.322995 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:47:52.322995 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:47:52.322995 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:47:52.322995 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:47:52.322995 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 19:47:52.322995 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:47:52.322995 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 93/183] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jun 23 19:47:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:47:52.323623 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1708f-5534-43c4-8593-f98576d92562 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:52.338506 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:52.355422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573ee80-2e2d-4347-8eaa-aaa93dce075a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:52.390638 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70783a14-2c48-46b5-9ce6-55c46efc57ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:52.391290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70783a14-2c48-46b5-9ce6-55c46efc57ce None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:52.391566 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70783a14-2c48-46b5-9ce6-55c46efc57ce None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:52.391838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70783a14-2c48-46b5-9ce6-55c46efc57ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:52.392751 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70783a14-2c48-46b5-9ce6-55c46efc57ce None admin] RBAC: Adding query filter params (role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb, scope.project.id=4c94dfce039f4ff29bec2f9831ec1748, user.id=5060168fc25744178d345904afb6c333) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:52.393063 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70783a14-2c48-46b5-9ce6-55c46efc57ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:52.394605 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70783a14-2c48-46b5-9ce6-55c46efc57ce None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:52.400963 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 91/184] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:47:52 2022] GET /identity/v3/role_assignments?user.id=5060168fc25744178d345904afb6c333&scope.project.id=4c94dfce039f4ff29bec2f9831ec1748&role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:52.402132 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc47ee6-3ecd-4b49-b66b-aa866837ce69 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:52.402412 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc47ee6-3ecd-4b49-b66b-aa866837ce69 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:52.403771 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc47ee6-3ecd-4b49-b66b-aa866837ce69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:52.429795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9bc47ee6-3ecd-4b49-b66b-aa866837ce69 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:52.431341 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9bc47ee6-3ecd-4b49-b66b-aa866837ce69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:52.432088 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade1708f-5534-43c4-8593-f98576d92562 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:52.470664 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4ed67-1eaa-440a-ba57-42fe269c0d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:52.471337 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4ed67-1eaa-440a-ba57-42fe269c0d7b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:52.471622 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4ed67-1eaa-440a-ba57-42fe269c0d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:52.471896 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4ed67-1eaa-440a-ba57-42fe269c0d7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:52.502428 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:52.520337 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 92/185] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:52.520337 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc47ee6-3ecd-4b49-b66b-aa866837ce69 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:52.556166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db4ed67-1eaa-440a-ba57-42fe269c0d7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:52.557568 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db4ed67-1eaa-440a-ba57-42fe269c0d7b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:52.568638 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:52.649327 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc47ee6-3ecd-4b49-b66b-aa866837ce69 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:52.669787 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 94/186] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:47:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:47:52.670489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4ed67-1eaa-440a-ba57-42fe269c0d7b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:52.684681 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0cc226-7022-4104-85ef-edb31576bf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:52.685441 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cc226-7022-4104-85ef-edb31576bf32 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:52.686000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cc226-7022-4104-85ef-edb31576bf32 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:52.686282 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cc226-7022-4104-85ef-edb31576bf32 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:52.687038 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0cc226-7022-4104-85ef-edb31576bf32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:52.687342 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0cc226-7022-4104-85ef-edb31576bf32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:52.689038 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0cc226-7022-4104-85ef-edb31576bf32 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:52.697062 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 93/187] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:47:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:52.698253 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258a477-effd-4669-9304-cd781c3b8083 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:52.698550 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258a477-effd-4669-9304-cd781c3b8083 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:52.698840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4258a477-effd-4669-9304-cd781c3b8083 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:52.703843 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 94/188] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:52.703843 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258a477-effd-4669-9304-cd781c3b8083 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:52.761268 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:52.770303 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4ed67-1eaa-440a-ba57-42fe269c0d7b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:52.779484 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:52.804068 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a5ba5-6318-45a1-b04f-1762859163c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:52.804521 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4258a477-effd-4669-9304-cd781c3b8083 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:52.804953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a5ba5-6318-45a1-b04f-1762859163c2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:52.805064 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a5ba5-6318-45a1-b04f-1762859163c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:52.805352 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a5ba5-6318-45a1-b04f-1762859163c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:52.806106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684a5ba5-6318-45a1-b04f-1762859163c2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:52.806409 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684a5ba5-6318-45a1-b04f-1762859163c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:52.807788 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684a5ba5-6318-45a1-b04f-1762859163c2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:52.816194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 95/189] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:47:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:52.817007 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a5ba5-6318-45a1-b04f-1762859163c2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:52.841817 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4532ff-3c9f-4372-8e4b-2a1908dfa50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:52.842596 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4532ff-3c9f-4372-8e4b-2a1908dfa50e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:52.842873 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4532ff-3c9f-4372-8e4b-2a1908dfa50e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:52.843151 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4532ff-3c9f-4372-8e4b-2a1908dfa50e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:52.844338 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4532ff-3c9f-4372-8e4b-2a1908dfa50e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:52.845825 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4532ff-3c9f-4372-8e4b-2a1908dfa50e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:52.847415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 95/190] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:47:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:52.848162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4532ff-3c9f-4372-8e4b-2a1908dfa50e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:52.925086 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a5ba5-6318-45a1-b04f-1762859163c2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:52.945748 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4532ff-3c9f-4372-8e4b-2a1908dfa50e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:52.960045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c547de-a809-405b-9967-d7f89a413f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:52.961447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c547de-a809-405b-9967-d7f89a413f15 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:52.961730 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c547de-a809-405b-9967-d7f89a413f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:52.962004 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c547de-a809-405b-9967-d7f89a413f15 None admin] PATH_INFO: `/v3/projects/5cfe3739530c48c29d910951147203ea` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:52.963194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c547de-a809-405b-9967-d7f89a413f15 None admin] RBAC: Authorizing `identity:get_project(project_id=5cfe3739530c48c29d910951147203ea)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:52.964678 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c547de-a809-405b-9967-d7f89a413f15 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:52.966382 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 96/191] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:47:52 2022] GET /identity/v3/projects/5cfe3739530c48c29d910951147203ea => generated 312 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:52.967509 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc2999-fb36-4a94-aac0-3e0ce6d30abe None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:52.967792 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc2999-fb36-4a94-aac0-3e0ce6d30abe None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:52.968067 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc2999-fb36-4a94-aac0-3e0ce6d30abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:52.974591 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:52.983874 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6acc2999-fb36-4a94-aac0-3e0ce6d30abe None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:52.984796 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1f206-725e-4e8c-a457-0298825b4ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:52.985089 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6acc2999-fb36-4a94-aac0-3e0ce6d30abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:52.985663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1f206-725e-4e8c-a457-0298825b4ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:52.985943 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1f206-725e-4e8c-a457-0298825b4ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:52.986213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1f206-725e-4e8c-a457-0298825b4ce9 None admin] PATH_INFO: `/v3/users/ff8ba25ac61f4edf86d1507803104750` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:53.000060 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d1f206-725e-4e8c-a457-0298825b4ce9 None admin] RBAC: Authorizing `identity:get_user(user_id=ff8ba25ac61f4edf86d1507803104750)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:53.001572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d1f206-725e-4e8c-a457-0298825b4ce9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:53.003216 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 96/192] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:47:52 2022] GET /identity/v3/users/ff8ba25ac61f4edf86d1507803104750 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:53.003966 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1f206-725e-4e8c-a457-0298825b4ce9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:53.014554 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:53.050951 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 97/193] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:53.051768 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc2999-fb36-4a94-aac0-3e0ce6d30abe None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:53.120008 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1f206-725e-4e8c-a457-0298825b4ce9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:53.149097 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc2999-fb36-4a94-aac0-3e0ce6d30abe None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:53.153202 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0357b5-1868-46e8-a450-5cc904328433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:53.154066 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0357b5-1868-46e8-a450-5cc904328433 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:53.154352 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0357b5-1868-46e8-a450-5cc904328433 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:53.154623 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0357b5-1868-46e8-a450-5cc904328433 None admin] PATH_INFO: `/v3/users/ff8ba25ac61f4edf86d1507803104750` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:53.155797 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0357b5-1868-46e8-a450-5cc904328433 None admin] RBAC: Authorizing `identity:get_user(user_id=ff8ba25ac61f4edf86d1507803104750)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:53.157282 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0357b5-1868-46e8-a450-5cc904328433 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:53.158894 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 97/194] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:47:53 2022] GET /identity/v3/users/ff8ba25ac61f4edf86d1507803104750 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:53.159730 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0357b5-1868-46e8-a450-5cc904328433 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:53.187291 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd3848-319c-4ae2-a04b-828285481698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:53.188211 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd3848-319c-4ae2-a04b-828285481698 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:53.188528 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd3848-319c-4ae2-a04b-828285481698 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:53.188804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd3848-319c-4ae2-a04b-828285481698 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:53.189592 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd3848-319c-4ae2-a04b-828285481698 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:53.191053 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd3848-319c-4ae2-a04b-828285481698 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:53.208943 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:53.255658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0357b5-1868-46e8-a450-5cc904328433 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:53.288006 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4236a31-1b81-4afa-bb50-aa68468abdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:53.288622 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4236a31-1b81-4afa-bb50-aa68468abdd0 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:53.288905 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4236a31-1b81-4afa-bb50-aa68468abdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:53.289178 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4236a31-1b81-4afa-bb50-aa68468abdd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:53.290007 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4236a31-1b81-4afa-bb50-aa68468abdd0 None admin] RBAC: Adding query filter params (role.id=860491bfbb574772a7a213d6a25962e0, scope.project.id=5cfe3739530c48c29d910951147203ea, user.id=fa6694f71dec4568a4b7724c3570eff4) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:53.290316 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4236a31-1b81-4afa-bb50-aa68468abdd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:53.291754 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4236a31-1b81-4afa-bb50-aa68468abdd0 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:53.297414 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 98/195] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:47:53 2022] GET /identity/v3/role_assignments?user.id=fa6694f71dec4568a4b7724c3570eff4&scope.project.id=5cfe3739530c48c29d910951147203ea&role.id=860491bfbb574772a7a213d6a25962e0 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:53.298171 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4236a31-1b81-4afa-bb50-aa68468abdd0 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:53.316384 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 98/196] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jun 23 19:47:53 2022] POST /identity/v3/users => generated 297 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:47:53.317479 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd3848-319c-4ae2-a04b-828285481698 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:53.355794 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:53.440228 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4236a31-1b81-4afa-bb50-aa68468abdd0 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:53.465931 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:53.476663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd3848-319c-4ae2-a04b-828285481698 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:53.486331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fc203-3651-4f09-b778-fb6e728dd9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:53.487332 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fc203-3651-4f09-b778-fb6e728dd9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:53.487897 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fc203-3651-4f09-b778-fb6e728dd9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:53.488170 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fc203-3651-4f09-b778-fb6e728dd9a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:53.526874 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e832ad-0cd5-4105-9930-bd44f44b9e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:53.527764 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e832ad-0cd5-4105-9930-bd44f44b9e59 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:53.528024 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e832ad-0cd5-4105-9930-bd44f44b9e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:53.528328 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e832ad-0cd5-4105-9930-bd44f44b9e59 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:53.543524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e832ad-0cd5-4105-9930-bd44f44b9e59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:47:53.551335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-a2e832ad-0cd5-4105-9930-bd44f44b9e59 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 19:47:53.551335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:47:53.551335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:47:53.551335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:47:53.551335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:47:53.551335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:47:53.551335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:47:53.551335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:47:53.551335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:47:53.551335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:47:53.551335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:47:53.551335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:47:53.551335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:47:53.551335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:47:53.551335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:47:53.551335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:47:53.552234 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:47:53.552234 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:47:53.552234 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:47:53.552234 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:47:53.552234 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:47:53.552234 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:47:53.552234 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:47:53.552234 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:47:53.552234 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:47:53.552234 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:47:53.552234 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:47:53.552234 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:47:53.552234 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:47:53.552234 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:47:53.552234 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:47:53.552234 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:47:53.553331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:47:53.553331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:47:53.553331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:47:53.553331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:47:53.553331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:47:53.553331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:47:53.553331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:47:53.553331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:47:53.553331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:47:53.553331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:47:53.553331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 19:47:53.553331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:47:53.555843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 99/197] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:47:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:47:53.556018 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e832ad-0cd5-4105-9930-bd44f44b9e59 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:53.577120 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fc203-3651-4f09-b778-fb6e728dd9a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:53.578761 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fc203-3651-4f09-b778-fb6e728dd9a1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:53.663894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:53.681336 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e832ad-0cd5-4105-9930-bd44f44b9e59 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:53.704621 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 99/198] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:47:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:47:53.705908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fc203-3651-4f09-b778-fb6e728dd9a1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:53.721916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:53.727526 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af968785-9b91-4226-bd66-98eecd700fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:53.728187 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af968785-9b91-4226-bd66-98eecd700fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:53.728502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af968785-9b91-4226-bd66-98eecd700fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:53.728782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af968785-9b91-4226-bd66-98eecd700fa0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:53.805221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af968785-9b91-4226-bd66-98eecd700fa0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:53.806636 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af968785-9b91-4226-bd66-98eecd700fa0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:53.810764 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fc203-3651-4f09-b778-fb6e728dd9a1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:53.847635 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc20ee9-210a-4602-8085-cb0538d97eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:53.848325 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc20ee9-210a-4602-8085-cb0538d97eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:53.848635 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc20ee9-210a-4602-8085-cb0538d97eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:53.848912 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc20ee9-210a-4602-8085-cb0538d97eb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:53.849783 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc20ee9-210a-4602-8085-cb0538d97eb1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:53.850098 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc20ee9-210a-4602-8085-cb0538d97eb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:53.852066 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc20ee9-210a-4602-8085-cb0538d97eb1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:53.861383 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 100/199] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jun 23 19:47:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:53.862040 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc20ee9-210a-4602-8085-cb0538d97eb1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:53.896852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:53.914242 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 100/200] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:47:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:47:53.918739 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af968785-9b91-4226-bd66-98eecd700fa0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:53.977845 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:53.988848 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc20ee9-210a-4602-8085-cb0538d97eb1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:54.028288 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af968785-9b91-4226-bd66-98eecd700fa0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:54.032373 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dcb50c-37ce-46d6-a658-6fa7fb32d98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:54.033562 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcb50c-37ce-46d6-a658-6fa7fb32d98c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:54.034032 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcb50c-37ce-46d6-a658-6fa7fb32d98c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:54.034489 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcb50c-37ce-46d6-a658-6fa7fb32d98c None admin] PATH_INFO: `/v3/projects/6c9c90447a664a9fa575e9c13ef585f0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:54.036454 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dcb50c-37ce-46d6-a658-6fa7fb32d98c None admin] RBAC: Authorizing `identity:get_project(project_id=6c9c90447a664a9fa575e9c13ef585f0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:54.039237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dcb50c-37ce-46d6-a658-6fa7fb32d98c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:54.041945 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 101/201] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:47:53 2022] GET /identity/v3/projects/6c9c90447a664a9fa575e9c13ef585f0 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:54.043419 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dcb50c-37ce-46d6-a658-6fa7fb32d98c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:54.075956 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd9ace-f295-424e-afd6-314f801bb3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:54.077022 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd9ace-f295-424e-afd6-314f801bb3e7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:54.077409 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd9ace-f295-424e-afd6-314f801bb3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:54.077784 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd9ace-f295-424e-afd6-314f801bb3e7 None admin] PATH_INFO: `/v3/users/f16b644eff4d40d092a047b8dc388112` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:54.079659 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd9ace-f295-424e-afd6-314f801bb3e7 None admin] RBAC: Authorizing `identity:get_user(user_id=f16b644eff4d40d092a047b8dc388112)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:54.081533 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd9ace-f295-424e-afd6-314f801bb3e7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:54.084572 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 101/202] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:47:53 2022] GET /identity/v3/users/f16b644eff4d40d092a047b8dc388112 => generated 289 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:54.086239 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd9ace-f295-424e-afd6-314f801bb3e7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:54.159711 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:54.164718 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dcb50c-37ce-46d6-a658-6fa7fb32d98c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:54.200576 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d3d87-5fc6-4f6e-95c3-e62961f3fd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:54.201738 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d3d87-5fc6-4f6e-95c3-e62961f3fd1f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:54.202162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d3d87-5fc6-4f6e-95c3-e62961f3fd1f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:54.202598 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d3d87-5fc6-4f6e-95c3-e62961f3fd1f None admin] PATH_INFO: `/v3/projects/6c9c90447a664a9fa575e9c13ef585f0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:54.203920 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d3d87-5fc6-4f6e-95c3-e62961f3fd1f None admin] RBAC: Authorizing `identity:get_project(project_id=6c9c90447a664a9fa575e9c13ef585f0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:54.205541 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d3d87-5fc6-4f6e-95c3-e62961f3fd1f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:54.207463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 102/203] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:47:54 2022] GET /identity/v3/projects/6c9c90447a664a9fa575e9c13ef585f0 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:54.208486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d3d87-5fc6-4f6e-95c3-e62961f3fd1f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:54.219465 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd9ace-f295-424e-afd6-314f801bb3e7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:54.225439 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:54.269351 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a96ca-b79c-49e3-9c2b-31eb19d953b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:54.270017 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a96ca-b79c-49e3-9c2b-31eb19d953b6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:54.270287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a96ca-b79c-49e3-9c2b-31eb19d953b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:54.270554 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a96ca-b79c-49e3-9c2b-31eb19d953b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:54.271875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a96ca-b79c-49e3-9c2b-31eb19d953b6 None admin] RBAC: Adding query filter params (role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb, scope.project.id=6c9c90447a664a9fa575e9c13ef585f0, user.id=ff8ba25ac61f4edf86d1507803104750) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:54.271875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a96ca-b79c-49e3-9c2b-31eb19d953b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:54.273840 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a96ca-b79c-49e3-9c2b-31eb19d953b6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:54.280083 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 102/204] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:47:54 2022] GET /identity/v3/role_assignments?user.id=ff8ba25ac61f4edf86d1507803104750&scope.project.id=6c9c90447a664a9fa575e9c13ef585f0&role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:54.289596 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a96ca-b79c-49e3-9c2b-31eb19d953b6 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:54.361363 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d3d87-5fc6-4f6e-95c3-e62961f3fd1f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:54.413686 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58552fb-fc24-4d33-b88f-192eca86bb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:54.414774 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58552fb-fc24-4d33-b88f-192eca86bb7f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:54.415287 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58552fb-fc24-4d33-b88f-192eca86bb7f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:54.415789 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58552fb-fc24-4d33-b88f-192eca86bb7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:54.453840 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a96ca-b79c-49e3-9c2b-31eb19d953b6 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:54.486917 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:54.528942 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620d498c-8809-42d2-bacc-a80e487cab99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:54.529987 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d498c-8809-42d2-bacc-a80e487cab99 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:54.532679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d498c-8809-42d2-bacc-a80e487cab99 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:54.533198 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d498c-8809-42d2-bacc-a80e487cab99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:54.534257 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620d498c-8809-42d2-bacc-a80e487cab99 None admin] RBAC: Adding query filter params (role.id=03e4cfc753754643ae08fa909647cfe3, scope.project.id=6c9c90447a664a9fa575e9c13ef585f0, user.id=ff8ba25ac61f4edf86d1507803104750) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:54.534784 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620d498c-8809-42d2-bacc-a80e487cab99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:54.540551 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620d498c-8809-42d2-bacc-a80e487cab99 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:54.549809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 103/205] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:47:54 2022] GET /identity/v3/role_assignments?user.id=ff8ba25ac61f4edf86d1507803104750&scope.project.id=6c9c90447a664a9fa575e9c13ef585f0&role.id=03e4cfc753754643ae08fa909647cfe3 => generated 259 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:54.583018 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58552fb-fc24-4d33-b88f-192eca86bb7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:54.584591 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58552fb-fc24-4d33-b88f-192eca86bb7f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:54.602921 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:54.767551 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 103/206] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:47:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:47:54.773266 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620d498c-8809-42d2-bacc-a80e487cab99 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:54.973515 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620d498c-8809-42d2-bacc-a80e487cab99 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:55.038641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ae4ea-8eca-4f3e-8d5b-22fbefbb52ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:55.039897 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ae4ea-8eca-4f3e-8d5b-22fbefbb52ff None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:55.040385 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ae4ea-8eca-4f3e-8d5b-22fbefbb52ff None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:55.040776 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ae4ea-8eca-4f3e-8d5b-22fbefbb52ff None admin] PATH_INFO: `/v3/projects/4c94dfce039f4ff29bec2f9831ec1748` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:55.043039 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ae4ea-8eca-4f3e-8d5b-22fbefbb52ff None admin] RBAC: Authorizing `identity:get_project(project_id=4c94dfce039f4ff29bec2f9831ec1748)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:55.051185 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42ae4ea-8eca-4f3e-8d5b-22fbefbb52ff None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:55.054903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 104/207] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:47:54 2022] GET /identity/v3/projects/4c94dfce039f4ff29bec2f9831ec1748 => generated 298 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:55.062376 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58552fb-fc24-4d33-b88f-192eca86bb7f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:55.223286 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58552fb-fc24-4d33-b88f-192eca86bb7f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:55.252674 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:55.266998 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b7000-e0c0-4602-a954-1b7730f729ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:55.267657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b7000-e0c0-4602-a954-1b7730f729ff None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:55.267934 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b7000-e0c0-4602-a954-1b7730f729ff None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:55.268208 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b7000-e0c0-4602-a954-1b7730f729ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:55.269150 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b7000-e0c0-4602-a954-1b7730f729ff None admin] RBAC: Adding query filter params (role.id=43f9a6d53a0b43c6ada3d7d2eea1c7d2, scope.project.id=4c94dfce039f4ff29bec2f9831ec1748, user.id=f16b644eff4d40d092a047b8dc388112) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:55.269459 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b7000-e0c0-4602-a954-1b7730f729ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:55.270916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b7000-e0c0-4602-a954-1b7730f729ff None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:55.277209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 104/208] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:47:55 2022] GET /identity/v3/role_assignments?user.id=f16b644eff4d40d092a047b8dc388112&scope.project.id=4c94dfce039f4ff29bec2f9831ec1748&role.id=43f9a6d53a0b43c6ada3d7d2eea1c7d2 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:55.876610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69fde4-ef4f-4959-8be4-401096a2b09e None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:55.877138 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69fde4-ef4f-4959-8be4-401096a2b09e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:55.877138 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69fde4-ef4f-4959-8be4-401096a2b09e None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:55.878584 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 105/209] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:55.885094 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65838d04-5c93-49dc-9b74-268daea3defd None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:55.885422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65838d04-5c93-49dc-9b74-268daea3defd None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:55.885646 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65838d04-5c93-49dc-9b74-268daea3defd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:55.906229 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-65838d04-5c93-49dc-9b74-268daea3defd None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:55.907425 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-65838d04-5c93-49dc-9b74-268daea3defd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:55.963923 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acf9bc-ae4d-4ca9-9ac3-338d90875366 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:55.964216 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acf9bc-ae4d-4ca9-9ac3-338d90875366 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:55.964697 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2acf9bc-ae4d-4ca9-9ac3-338d90875366 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:55.967936 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 106/210] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:55.973999 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e47720-9e19-4709-b4be-2f6598232d72 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:55.974282 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e47720-9e19-4709-b4be-2f6598232d72 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:55.974560 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e47720-9e19-4709-b4be-2f6598232d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:55.995001 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 105/211] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:55.999807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-38e47720-9e19-4709-b4be-2f6598232d72 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:55.999807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-38e47720-9e19-4709-b4be-2f6598232d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:56.025100 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65838d04-5c93-49dc-9b74-268daea3defd None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:56.047105 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:56.075814 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 107/212] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 19:47:56.101930 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e47720-9e19-4709-b4be-2f6598232d72 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:56.147394 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65838d04-5c93-49dc-9b74-268daea3defd None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:56.156914 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:56.190541 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94818773-51dd-4726-8d82-856ccd2d4fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:56.191504 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94818773-51dd-4726-8d82-856ccd2d4fbb None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:56.192107 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94818773-51dd-4726-8d82-856ccd2d4fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:56.193040 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94818773-51dd-4726-8d82-856ccd2d4fbb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:56.206453 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94818773-51dd-4726-8d82-856ccd2d4fbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:47:56.212536 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e47720-9e19-4709-b4be-2f6598232d72 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:56.216599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-94818773-51dd-4726-8d82-856ccd2d4fbb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:47:56.216599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:47:56.216599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:47:56.216599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:47:56.216599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:47:56.216599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:47:56.216599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:47:56.216599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:47:56.216599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:47:56.216599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:47:56.216599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:47:56.216599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:47:56.216599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:47:56.216599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:47:56.216599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:47:56.216599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:47:56.217637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:47:56.217637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:47:56.217637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:47:56.217637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:47:56.217637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:47:56.217637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:47:56.217637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:47:56.217637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:47:56.217637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:47:56.217637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:47:56.217637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:47:56.217637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:47:56.217637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:47:56.217637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:47:56.217637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:47:56.217637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:47:56.218615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:47:56.218615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:47:56.218615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:47:56.218615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:47:56.218615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:47:56.218615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:47:56.218615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:47:56.218615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:47:56.218615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:47:56.218615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:47:56.218615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:47:56.218615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:47:56.219911 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 106/213] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:47:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:47:56.225771 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94818773-51dd-4726-8d82-856ccd2d4fbb None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:56.261540 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ba8ab-b50c-4f39-bcd8-d670b76bb848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:56.262602 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ba8ab-b50c-4f39-bcd8-d670b76bb848 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:56.263057 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ba8ab-b50c-4f39-bcd8-d670b76bb848 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:56.263500 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ba8ab-b50c-4f39-bcd8-d670b76bb848 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:56.264860 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ba8ab-b50c-4f39-bcd8-d670b76bb848 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:56.266758 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ba8ab-b50c-4f39-bcd8-d670b76bb848 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:56.268692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 108/214] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:47:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:56.274194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ba8ab-b50c-4f39-bcd8-d670b76bb848 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:56.326885 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94818773-51dd-4726-8d82-856ccd2d4fbb None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:56.329511 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:56.360684 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b982fd27-0a4d-4477-9aa8-005dc3490db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:56.361355 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982fd27-0a4d-4477-9aa8-005dc3490db5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:56.361924 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982fd27-0a4d-4477-9aa8-005dc3490db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:56.361924 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982fd27-0a4d-4477-9aa8-005dc3490db5 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:56.362694 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b982fd27-0a4d-4477-9aa8-005dc3490db5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:56.362975 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b982fd27-0a4d-4477-9aa8-005dc3490db5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:56.364454 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b982fd27-0a4d-4477-9aa8-005dc3490db5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:56.377917 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 107/215] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jun 23 19:47:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:56.380979 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b982fd27-0a4d-4477-9aa8-005dc3490db5 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:56.393180 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ba8ab-b50c-4f39-bcd8-d670b76bb848 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:56.408506 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:56.432809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b698662-4a42-46fd-922f-c0b1479e3fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:56.433967 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b698662-4a42-46fd-922f-c0b1479e3fd6 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:56.434452 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b698662-4a42-46fd-922f-c0b1479e3fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:56.434926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b698662-4a42-46fd-922f-c0b1479e3fd6 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:56.435850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b698662-4a42-46fd-922f-c0b1479e3fd6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:56.437683 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b698662-4a42-46fd-922f-c0b1479e3fd6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:56.488109 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b982fd27-0a4d-4477-9aa8-005dc3490db5 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:56.523843 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eab4f8-a015-4b9d-b8bc-c2669c29f0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:56.524745 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eab4f8-a015-4b9d-b8bc-c2669c29f0dd None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:56.525026 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eab4f8-a015-4b9d-b8bc-c2669c29f0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:56.525295 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eab4f8-a015-4b9d-b8bc-c2669c29f0dd None admin] PATH_INFO: `/v3/users/9085a6e690aa4aacb85c0f3ff27767dc` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:56.539653 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eab4f8-a015-4b9d-b8bc-c2669c29f0dd None admin] RBAC: Authorizing `identity:get_user(user_id=9085a6e690aa4aacb85c0f3ff27767dc)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:56.547534 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64eab4f8-a015-4b9d-b8bc-c2669c29f0dd None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:56.547534 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 108/216] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:47:56 2022] GET /identity/v3/users/9085a6e690aa4aacb85c0f3ff27767dc => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:56.548331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 109/217] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jun 23 19:47:56 2022] POST /identity/v3/users => generated 297 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:47:56.552827 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eab4f8-a015-4b9d-b8bc-c2669c29f0dd None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:56.654051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:56.671701 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eab4f8-a015-4b9d-b8bc-c2669c29f0dd None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:56.712719 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d41339-94cf-4f65-b7c0-cf71455032f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:56.713725 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d41339-94cf-4f65-b7c0-cf71455032f3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:56.714181 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d41339-94cf-4f65-b7c0-cf71455032f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:56.714650 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d41339-94cf-4f65-b7c0-cf71455032f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:56.787600 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d41339-94cf-4f65-b7c0-cf71455032f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:56.789895 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d41339-94cf-4f65-b7c0-cf71455032f3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:56.834091 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31696003-5b49-4381-b1e5-b7a9bebc7625 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:56.834630 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31696003-5b49-4381-b1e5-b7a9bebc7625 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:56.835095 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31696003-5b49-4381-b1e5-b7a9bebc7625 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:56.836736 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 110/218] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:56.843409 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f2571-98ef-4f02-82b1-af4af7b2c8de None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:56.843929 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f2571-98ef-4f02-82b1-af4af7b2c8de None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:56.844507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f2571-98ef-4f02-82b1-af4af7b2c8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:56.866883 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5d6f2571-98ef-4f02-82b1-af4af7b2c8de None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:56.868097 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5d6f2571-98ef-4f02-82b1-af4af7b2c8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:56.886593 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:56.908464 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 109/219] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:47:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:47:56.913491 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d41339-94cf-4f65-b7c0-cf71455032f3 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:56.963765 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 111/220] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 19:47:56.966222 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deeb268-9ca5-4b73-a9f2-f0fa2ddfadd3 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:56.966717 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deeb268-9ca5-4b73-a9f2-f0fa2ddfadd3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:56.967211 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deeb268-9ca5-4b73-a9f2-f0fa2ddfadd3 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:56.969649 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 112/221] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:56.975053 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860135e0-01d7-400f-a050-85f8898084c5 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:56.976088 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860135e0-01d7-400f-a050-85f8898084c5 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:56.976744 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860135e0-01d7-400f-a050-85f8898084c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:56.998719 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-860135e0-01d7-400f-a050-85f8898084c5 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:57.000592 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-860135e0-01d7-400f-a050-85f8898084c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:57.036922 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d41339-94cf-4f65-b7c0-cf71455032f3 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:57.042461 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:57.074055 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce887d84-318c-42dc-86b1-4df430517cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:57.075287 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce887d84-318c-42dc-86b1-4df430517cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:57.075768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce887d84-318c-42dc-86b1-4df430517cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:57.076238 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce887d84-318c-42dc-86b1-4df430517cf3 None admin] PATH_INFO: `/v3/projects/6c9c90447a664a9fa575e9c13ef585f0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:57.077688 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce887d84-318c-42dc-86b1-4df430517cf3 None admin] RBAC: Authorizing `identity:get_project(project_id=6c9c90447a664a9fa575e9c13ef585f0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:57.079365 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce887d84-318c-42dc-86b1-4df430517cf3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:57.081614 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 110/222] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:47:56 2022] GET /identity/v3/projects/6c9c90447a664a9fa575e9c13ef585f0 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:47:57.082593 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce887d84-318c-42dc-86b1-4df430517cf3 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:57.097583 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 113/223] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 19:47:57.098591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860135e0-01d7-400f-a050-85f8898084c5 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:57.141971 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:57.189483 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce887d84-318c-42dc-86b1-4df430517cf3 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:57.206954 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860135e0-01d7-400f-a050-85f8898084c5 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:57.226112 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb4063-aa3a-4a7b-b469-690030a3ee55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:57.227183 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb4063-aa3a-4a7b-b469-690030a3ee55 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:57.227642 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb4063-aa3a-4a7b-b469-690030a3ee55 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:57.228090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb4063-aa3a-4a7b-b469-690030a3ee55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:57.239250 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb4063-aa3a-4a7b-b469-690030a3ee55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:47:57.243150 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108133d-3487-47c2-8388-b88e9fbef9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:57.243890 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108133d-3487-47c2-8388-b88e9fbef9f5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:57.244291 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108133d-3487-47c2-8388-b88e9fbef9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:57.244827 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108133d-3487-47c2-8388-b88e9fbef9f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:57.245829 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4108133d-3487-47c2-8388-b88e9fbef9f5 None admin] RBAC: Adding query filter params (role.id=860491bfbb574772a7a213d6a25962e0, scope.project.id=6c9c90447a664a9fa575e9c13ef585f0, user.id=9085a6e690aa4aacb85c0f3ff27767dc) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:57.246625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4108133d-3487-47c2-8388-b88e9fbef9f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:57.247920 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4108133d-3487-47c2-8388-b88e9fbef9f5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:57.249208 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-85cb4063-aa3a-4a7b-b469-690030a3ee55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:47:57.249208 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:47:57.249208 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:47:57.249208 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:47:57.249208 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:47:57.249208 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:47:57.249208 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:47:57.249208 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:47:57.249208 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:47:57.249208 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:47:57.249208 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:47:57.249208 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:47:57.249208 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:47:57.249208 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:47:57.249208 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:47:57.249208 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:47:57.250078 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:47:57.250078 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:47:57.250078 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:47:57.250078 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:47:57.250078 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:47:57.250078 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:47:57.250078 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:47:57.250078 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:47:57.250078 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:47:57.250078 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:47:57.250078 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:47:57.250078 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:47:57.250078 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:47:57.250078 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:47:57.250078 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:47:57.250078 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:47:57.250858 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:47:57.250858 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:47:57.250858 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:47:57.250858 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:47:57.250858 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:47:57.250858 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:47:57.250858 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:47:57.250858 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:47:57.250858 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:47:57.250858 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:47:57.250858 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:47:57.250858 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:47:57.253212 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 111/224] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:47:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:47:57.254209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb4063-aa3a-4a7b-b469-690030a3ee55 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:57.259503 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 114/225] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:47:57 2022] GET /identity/v3/role_assignments?user.id=9085a6e690aa4aacb85c0f3ff27767dc&scope.project.id=6c9c90447a664a9fa575e9c13ef585f0&role.id=860491bfbb574772a7a213d6a25962e0 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:57.260427 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108133d-3487-47c2-8388-b88e9fbef9f5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:57.313056 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:57.436768 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108133d-3487-47c2-8388-b88e9fbef9f5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:57.443749 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb4063-aa3a-4a7b-b469-690030a3ee55 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:57.460933 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:57.495836 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47757315-43d7-4d3d-be10-be75d4f5a20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:57.496618 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47757315-43d7-4d3d-be10-be75d4f5a20a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:57.496897 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47757315-43d7-4d3d-be10-be75d4f5a20a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:57.497167 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47757315-43d7-4d3d-be10-be75d4f5a20a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:57.503416 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3374cf-662e-490b-8dcf-dc55752cb887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:57.504087 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3374cf-662e-490b-8dcf-dc55752cb887 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:57.504388 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3374cf-662e-490b-8dcf-dc55752cb887 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:57.504660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3374cf-662e-490b-8dcf-dc55752cb887 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:57.505416 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3374cf-662e-490b-8dcf-dc55752cb887 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:57.505723 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3374cf-662e-490b-8dcf-dc55752cb887 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:57.507925 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3374cf-662e-490b-8dcf-dc55752cb887 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:57.521016 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47757315-43d7-4d3d-be10-be75d4f5a20a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:47:57.529682 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-47757315-43d7-4d3d-be10-be75d4f5a20a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 19:47:57.529682 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:47:57.529682 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:47:57.529682 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:47:57.529682 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:47:57.529682 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:47:57.529682 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:47:57.529682 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:47:57.529682 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:47:57.529682 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:47:57.529682 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:47:57.529682 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:47:57.529682 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:47:57.529682 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:47:57.529682 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:47:57.529682 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:47:57.530538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:47:57.530538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:47:57.530538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:47:57.530538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:47:57.530538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:47:57.530538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:47:57.530538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:47:57.530538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:47:57.530538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:47:57.530538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:47:57.530538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:47:57.530538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:47:57.530538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:47:57.530538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:47:57.530538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:47:57.530538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:47:57.531320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:47:57.531320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:47:57.531320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:47:57.531320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:47:57.531320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:47:57.531320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:47:57.531320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:47:57.531320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:47:57.531320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:47:57.531320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:47:57.531320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 19:47:57.531320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:47:57.531320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 112/226] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jun 23 19:47:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:47:57.535012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 115/227] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:47:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:57.541784 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47757315-43d7-4d3d-be10-be75d4f5a20a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:57.559410 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3374cf-662e-490b-8dcf-dc55752cb887 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:57.662722 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47757315-43d7-4d3d-be10-be75d4f5a20a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:57.699605 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb2e71-4882-4b5e-9c95-0f115e639db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:57.700282 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb2e71-4882-4b5e-9c95-0f115e639db8 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:57.700591 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb2e71-4882-4b5e-9c95-0f115e639db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:57.700874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb2e71-4882-4b5e-9c95-0f115e639db8 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:57.701632 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cb2e71-4882-4b5e-9c95-0f115e639db8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:57.701942 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cb2e71-4882-4b5e-9c95-0f115e639db8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:57.703348 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cb2e71-4882-4b5e-9c95-0f115e639db8 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:57.712329 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 113/228] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:47:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:57.718708 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb2e71-4882-4b5e-9c95-0f115e639db8 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:57.774370 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:57.784640 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:57.800084 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3374cf-662e-490b-8dcf-dc55752cb887 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:57.846156 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150e196-9e9c-4847-b7e0-76c194d4407f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:57.847040 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150e196-9e9c-4847-b7e0-76c194d4407f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:57.847314 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150e196-9e9c-4847-b7e0-76c194d4407f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:57.847578 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150e196-9e9c-4847-b7e0-76c194d4407f None admin] PATH_INFO: `/v3/users/ff8ba25ac61f4edf86d1507803104750` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:57.850220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f150e196-9e9c-4847-b7e0-76c194d4407f None admin] RBAC: Authorizing `identity:get_user(user_id=ff8ba25ac61f4edf86d1507803104750)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:57.851785 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f150e196-9e9c-4847-b7e0-76c194d4407f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:57.854038 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 116/229] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:47:57 2022] GET /identity/v3/users/ff8ba25ac61f4edf86d1507803104750 => generated 283 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:57.860385 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150e196-9e9c-4847-b7e0-76c194d4407f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:57.923510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb2e71-4882-4b5e-9c95-0f115e639db8 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:57.960105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd6531-ba57-48fa-8b84-c4e8f2b27eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:57.961011 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd6531-ba57-48fa-8b84-c4e8f2b27eeb None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:57.961296 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd6531-ba57-48fa-8b84-c4e8f2b27eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:57.961589 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd6531-ba57-48fa-8b84-c4e8f2b27eeb None admin] PATH_INFO: `/v3/users/ff8ba25ac61f4edf86d1507803104750` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:57.962883 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd6531-ba57-48fa-8b84-c4e8f2b27eeb None admin] RBAC: Authorizing `identity:get_user(user_id=ff8ba25ac61f4edf86d1507803104750)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:57.964424 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd6531-ba57-48fa-8b84-c4e8f2b27eeb None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:57.966067 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 114/230] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:47:57 2022] GET /identity/v3/users/ff8ba25ac61f4edf86d1507803104750 => generated 283 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:47:57.977266 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd6531-ba57-48fa-8b84-c4e8f2b27eeb None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:57.996034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150e196-9e9c-4847-b7e0-76c194d4407f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:58.068543 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61216e92-8bdb-40bf-a42b-8babee3a68bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:58.069747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61216e92-8bdb-40bf-a42b-8babee3a68bd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:58.070179 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61216e92-8bdb-40bf-a42b-8babee3a68bd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:58.071573 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61216e92-8bdb-40bf-a42b-8babee3a68bd None admin] PATH_INFO: `/v3/projects/6c9c90447a664a9fa575e9c13ef585f0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:58.073277 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61216e92-8bdb-40bf-a42b-8babee3a68bd None admin] RBAC: Authorizing `identity:get_project(project_id=6c9c90447a664a9fa575e9c13ef585f0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:58.077441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61216e92-8bdb-40bf-a42b-8babee3a68bd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:58.082386 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 117/231] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:47:57 2022] GET /identity/v3/projects/6c9c90447a664a9fa575e9c13ef585f0 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:47:58.089179 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61216e92-8bdb-40bf-a42b-8babee3a68bd None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:58.140325 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd6531-ba57-48fa-8b84-c4e8f2b27eeb None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:58.140729 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:58.152070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:58.173903 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12956b73-1350-4a32-8db7-622c032e04c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:58.174905 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12956b73-1350-4a32-8db7-622c032e04c0 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:58.175349 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12956b73-1350-4a32-8db7-622c032e04c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:58.175666 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12956b73-1350-4a32-8db7-622c032e04c0 None admin] PATH_INFO: `/v3/projects/6c9c90447a664a9fa575e9c13ef585f0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:58.177775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12956b73-1350-4a32-8db7-622c032e04c0 None admin] RBAC: Authorizing `identity:get_project(project_id=6c9c90447a664a9fa575e9c13ef585f0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:58.184346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12956b73-1350-4a32-8db7-622c032e04c0 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:58.184346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 115/232] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:47:57 2022] GET /identity/v3/projects/6c9c90447a664a9fa575e9c13ef585f0 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:58.190278 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12956b73-1350-4a32-8db7-622c032e04c0 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:58.306436 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12956b73-1350-4a32-8db7-622c032e04c0 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:58.311362 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61216e92-8bdb-40bf-a42b-8babee3a68bd None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:58.342389 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edcf5f7-cf9b-4a65-a3aa-90240fc38d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:58.343216 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcf5f7-cf9b-4a65-a3aa-90240fc38d95 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:58.343504 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcf5f7-cf9b-4a65-a3aa-90240fc38d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:58.343781 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcf5f7-cf9b-4a65-a3aa-90240fc38d95 None admin] PATH_INFO: `/v3/projects/6c9c90447a664a9fa575e9c13ef585f0/users/ff8ba25ac61f4edf86d1507803104750/roles/03e4cfc753754643ae08fa909647cfe3` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:58.346099 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edcf5f7-cf9b-4a65-a3aa-90240fc38d95 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c9c90447a664a9fa575e9c13ef585f0, user_id=ff8ba25ac61f4edf86d1507803104750, role_id=03e4cfc753754643ae08fa909647cfe3)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:58.347769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edcf5f7-cf9b-4a65-a3aa-90240fc38d95 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:58.372447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95258b45-628f-4db8-a328-af279113b48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:58.373340 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95258b45-628f-4db8-a328-af279113b48b None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:58.373622 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95258b45-628f-4db8-a328-af279113b48b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:58.373983 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95258b45-628f-4db8-a328-af279113b48b None admin] PATH_INFO: `/v3/projects/6c9c90447a664a9fa575e9c13ef585f0/users/ff8ba25ac61f4edf86d1507803104750/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:58.376117 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95258b45-628f-4db8-a328-af279113b48b None admin] RBAC: Authorizing `identity:create_grant(project_id=6c9c90447a664a9fa575e9c13ef585f0, user_id=ff8ba25ac61f4edf86d1507803104750, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:58.377653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95258b45-628f-4db8-a328-af279113b48b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:58.440973 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 116/233] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jun 23 19:47:58 2022] PUT /identity/v3/projects/6c9c90447a664a9fa575e9c13ef585f0/users/ff8ba25ac61f4edf86d1507803104750/roles/03e4cfc753754643ae08fa909647cfe3 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:47:58.487206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 118/234] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jun 23 19:47:58 2022] PUT /identity/v3/projects/6c9c90447a664a9fa575e9c13ef585f0/users/ff8ba25ac61f4edf86d1507803104750/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:47:58.990401 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad22b1-8423-4875-831a-4c6a43b69044 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:58.990977 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad22b1-8423-4875-831a-4c6a43b69044 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:58.991466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad22b1-8423-4875-831a-4c6a43b69044 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:58.993080 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 117/235] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:47:58.998428 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fc51d-e6c2-469d-8a01-74bbc1a826cc None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:58.999293 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fc51d-e6c2-469d-8a01-74bbc1a826cc None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:58.999739 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fc51d-e6c2-469d-8a01-74bbc1a826cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:59.020999 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1d7fc51d-e6c2-469d-8a01-74bbc1a826cc None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:59.023012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1d7fc51d-e6c2-469d-8a01-74bbc1a826cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:59.108012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 119/236] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:59.132653 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad22b1-8423-4875-831a-4c6a43b69044 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:59.186576 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:59.290969 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad22b1-8423-4875-831a-4c6a43b69044 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:59.307775 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd8829-2ca0-42eb-a640-6d57cc48e25e None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:59.308404 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd8829-2ca0-42eb-a640-6d57cc48e25e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:59.308880 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd8829-2ca0-42eb-a640-6d57cc48e25e None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:59.310456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 120/237] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:47:59.315817 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed01bd-c200-4745-8793-ec6fddb0196c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:59.316107 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed01bd-c200-4745-8793-ec6fddb0196c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:59.316419 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed01bd-c200-4745-8793-ec6fddb0196c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:59.333702 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad3bbc-55bb-4060-96a9-4f7dca202d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:59.335371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad3bbc-55bb-4060-96a9-4f7dca202d66 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:59.335854 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad3bbc-55bb-4060-96a9-4f7dca202d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:59.336373 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad3bbc-55bb-4060-96a9-4f7dca202d66 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:59.337756 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ad3bbc-55bb-4060-96a9-4f7dca202d66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:59.339436 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ad3bbc-55bb-4060-96a9-4f7dca202d66 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:59.341621 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 118/238] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:47:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:47:59.351062 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad3bbc-55bb-4060-96a9-4f7dca202d66 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:59.351248 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-38ed01bd-c200-4745-8793-ec6fddb0196c None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:59.352682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-38ed01bd-c200-4745-8793-ec6fddb0196c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:59.370678 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:59.519123 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 121/239] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:47:59.572323 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:59.574491 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad3bbc-55bb-4060-96a9-4f7dca202d66 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:59.591607 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed01bd-c200-4745-8793-ec6fddb0196c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:59.614261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439b4c8b-2da4-47dd-abfd-cba4a5c9f7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:59.617223 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b4c8b-2da4-47dd-abfd-cba4a5c9f7c2 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:59.617713 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b4c8b-2da4-47dd-abfd-cba4a5c9f7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:59.618184 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b4c8b-2da4-47dd-abfd-cba4a5c9f7c2 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:59.619139 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439b4c8b-2da4-47dd-abfd-cba4a5c9f7c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:59.620827 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439b4c8b-2da4-47dd-abfd-cba4a5c9f7c2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:47:59.715077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 119/240] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jun 23 19:47:59 2022] POST /identity/v3/users => generated 293 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:47:59.728916 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed01bd-c200-4745-8793-ec6fddb0196c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:59.781204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b151ec8-87f1-4ceb-9003-b527903e16bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:59.782405 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b151ec8-87f1-4ceb-9003-b527903e16bd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:59.782898 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b151ec8-87f1-4ceb-9003-b527903e16bd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:59.785264 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b151ec8-87f1-4ceb-9003-b527903e16bd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:59.800770 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b151ec8-87f1-4ceb-9003-b527903e16bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:47:59.811635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-7b151ec8-87f1-4ceb-9003-b527903e16bd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 19:47:59.811635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:47:59.811635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:47:59.811635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:47:59.811635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:47:59.811635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:47:59.811635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:47:59.811635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:47:59.811635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:47:59.811635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:47:59.811635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:47:59.811635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:47:59.811635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:47:59.811635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:47:59.811635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:47:59.811635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:47:59.812569 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:47:59.812569 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:47:59.812569 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:47:59.812569 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:47:59.812569 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:47:59.812569 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:47:59.812569 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:47:59.812569 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:47:59.812569 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:47:59.812569 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:47:59.812569 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:47:59.812569 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:47:59.812569 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:47:59.812569 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:47:59.812569 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:47:59.812569 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:47:59.813642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:47:59.813642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:47:59.813642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:47:59.813642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:47:59.813642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:47:59.813642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:47:59.813642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:47:59.813642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:47:59.813642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:47:59.813642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:47:59.813642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 19:47:59.813642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:47:59.816052 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 122/241] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:47:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:47:59.822470 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439b4c8b-2da4-47dd-abfd-cba4a5c9f7c2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:47:59.829242 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd106c3-f37b-46eb-8f40-54170284f9cd None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:59.829534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd106c3-f37b-46eb-8f40-54170284f9cd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:59.829818 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd106c3-f37b-46eb-8f40-54170284f9cd None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:59.831228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 123/242] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:47:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:47:59.837854 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215686d9-b43a-48b9-8f3e-5445e2674a78 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:59.838156 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215686d9-b43a-48b9-8f3e-5445e2674a78 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:59.838418 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215686d9-b43a-48b9-8f3e-5445e2674a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:59.845727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:47:59.877533 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-215686d9-b43a-48b9-8f3e-5445e2674a78 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:47:59.878777 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-215686d9-b43a-48b9-8f3e-5445e2674a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:47:59.949869 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439b4c8b-2da4-47dd-abfd-cba4a5c9f7c2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:47:59.985889 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 124/243] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:47:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 23 19:47:59.988880 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efcc97-d5c6-4532-8407-309fd8d427ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:47:59.989584 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efcc97-d5c6-4532-8407-309fd8d427ab None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:47:59.989935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efcc97-d5c6-4532-8407-309fd8d427ab None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:47:59.990276 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efcc97-d5c6-4532-8407-309fd8d427ab None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:47:59.991672 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5efcc97-d5c6-4532-8407-309fd8d427ab None admin] RBAC: Adding query filter params (name=reader) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:47:59.992043 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5efcc97-d5c6-4532-8407-309fd8d427ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:47:59.994187 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5efcc97-d5c6-4532-8407-309fd8d427ab None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:00.002584 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 120/244] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jun 23 19:47:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:00.008340 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215686d9-b43a-48b9-8f3e-5445e2674a78 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:00.010408 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efcc97-d5c6-4532-8407-309fd8d427ab None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:00.033608 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:00.144585 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215686d9-b43a-48b9-8f3e-5445e2674a78 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:00.151351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efcc97-d5c6-4532-8407-309fd8d427ab None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:00.204993 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c0b9c-bab5-4f31-a91b-cd5eecb1c19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:00.206368 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c0b9c-bab5-4f31-a91b-cd5eecb1c19a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:00.206669 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c0b9c-bab5-4f31-a91b-cd5eecb1c19a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:00.207088 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c0b9c-bab5-4f31-a91b-cd5eecb1c19a None admin] PATH_INFO: `/v3/users/e67cc18cbc114cd3ba4c38bb9ebb91c8` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:00.209630 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:00.224809 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71130144-1e92-43b6-bda6-31dad663c0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:00.226555 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71130144-1e92-43b6-bda6-31dad663c0fb None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:00.228505 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71130144-1e92-43b6-bda6-31dad663c0fb None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:00.228505 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71130144-1e92-43b6-bda6-31dad663c0fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:00.241931 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c0b9c-bab5-4f31-a91b-cd5eecb1c19a None admin] RBAC: Authorizing `identity:get_user(user_id=e67cc18cbc114cd3ba4c38bb9ebb91c8)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:00.243431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c0b9c-bab5-4f31-a91b-cd5eecb1c19a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:00.247402 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 125/245] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:48:00 2022] GET /identity/v3/users/e67cc18cbc114cd3ba4c38bb9ebb91c8 => generated 297 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:00.252760 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c0b9c-bab5-4f31-a91b-cd5eecb1c19a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:00.258936 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71130144-1e92-43b6-bda6-31dad663c0fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:00.269320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-71130144-1e92-43b6-bda6-31dad663c0fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:48:00.269320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:00.269320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:00.269320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:00.269320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:00.269320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:00.269320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:00.269320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:00.269320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:00.269320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:00.269320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:00.269320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:00.269320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:00.269320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:00.269320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:00.269320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:00.270219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:00.270219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:00.270219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:00.270219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:00.270219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:00.270219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:00.270219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:00.270219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:00.270219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:00.270219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:00.270219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:00.270219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:00.270219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:00.270219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:00.270219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:00.270219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:00.271012 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:00.271012 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:00.271012 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:00.271012 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:00.271012 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:00.271012 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:00.271012 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:00.271012 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:00.271012 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:00.271012 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:00.271012 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:48:00.271012 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:00.273302 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 121/246] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:48:00.280119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71130144-1e92-43b6-bda6-31dad663c0fb None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:00.364098 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c0b9c-bab5-4f31-a91b-cd5eecb1c19a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:00.372754 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:00.397118 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71130144-1e92-43b6-bda6-31dad663c0fb None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:00.401937 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecdec37-22dd-4fba-8145-8e7f535cbd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:00.402711 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdec37-22dd-4fba-8145-8e7f535cbd6a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:00.403096 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdec37-22dd-4fba-8145-8e7f535cbd6a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:00.403473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecdec37-22dd-4fba-8145-8e7f535cbd6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:00.446363 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf20aec-bd7d-4a0b-a5f9-e4675233648a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:00.447290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf20aec-bd7d-4a0b-a5f9-e4675233648a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:00.447714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf20aec-bd7d-4a0b-a5f9-e4675233648a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:00.448148 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf20aec-bd7d-4a0b-a5f9-e4675233648a None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:00.449103 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf20aec-bd7d-4a0b-a5f9-e4675233648a None admin] RBAC: Adding query filter params (name=member) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:00.449558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf20aec-bd7d-4a0b-a5f9-e4675233648a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:00.451129 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf20aec-bd7d-4a0b-a5f9-e4675233648a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:00.464347 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 122/247] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jun 23 19:48:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:00.473308 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf20aec-bd7d-4a0b-a5f9-e4675233648a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:00.487618 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecdec37-22dd-4fba-8145-8e7f535cbd6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:00.489353 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecdec37-22dd-4fba-8145-8e7f535cbd6a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:00.494091 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:00.596670 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 126/248] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:48:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:48:00.602984 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecdec37-22dd-4fba-8145-8e7f535cbd6a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:00.665235 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf20aec-bd7d-4a0b-a5f9-e4675233648a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:00.702326 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:00.711918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecdec37-22dd-4fba-8145-8e7f535cbd6a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:00.742040 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf5574-f2df-4edb-9f1d-9c37b3b26ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:00.742938 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf5574-f2df-4edb-9f1d-9c37b3b26ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:00.743216 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf5574-f2df-4edb-9f1d-9c37b3b26ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:00.743486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf5574-f2df-4edb-9f1d-9c37b3b26ea8 None admin] PATH_INFO: `/v3/users/9085a6e690aa4aacb85c0f3ff27767dc` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:00.745006 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf5574-f2df-4edb-9f1d-9c37b3b26ea8 None admin] RBAC: Authorizing `identity:get_user(user_id=9085a6e690aa4aacb85c0f3ff27767dc)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:00.746482 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf5574-f2df-4edb-9f1d-9c37b3b26ea8 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:00.748141 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 123/249] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:48:00 2022] GET /identity/v3/users/9085a6e690aa4aacb85c0f3ff27767dc => generated 297 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:00.749477 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba82298-b68d-48f6-acba-4433508c5649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:00.750363 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba82298-b68d-48f6-acba-4433508c5649 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:00.750646 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba82298-b68d-48f6-acba-4433508c5649 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:00.750922 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba82298-b68d-48f6-acba-4433508c5649 None admin] PATH_INFO: `/v3/projects/6c9c90447a664a9fa575e9c13ef585f0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:00.752137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba82298-b68d-48f6-acba-4433508c5649 None admin] RBAC: Authorizing `identity:get_project(project_id=6c9c90447a664a9fa575e9c13ef585f0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:00.753763 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba82298-b68d-48f6-acba-4433508c5649 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:00.755159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf5574-f2df-4edb-9f1d-9c37b3b26ea8 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:00.755765 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 127/250] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:48:00 2022] GET /identity/v3/projects/6c9c90447a664a9fa575e9c13ef585f0 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:00.761966 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba82298-b68d-48f6-acba-4433508c5649 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:00.778282 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:00.918483 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba82298-b68d-48f6-acba-4433508c5649 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:00.945246 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf5574-f2df-4edb-9f1d-9c37b3b26ea8 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:00.967386 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b51a4e-0f97-454b-bac7-d268a7f391ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:00.968053 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b51a4e-0f97-454b-bac7-d268a7f391ce None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:00.968357 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b51a4e-0f97-454b-bac7-d268a7f391ce None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:00.970088 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b51a4e-0f97-454b-bac7-d268a7f391ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:00.970943 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b51a4e-0f97-454b-bac7-d268a7f391ce None admin] RBAC: Adding query filter params (role.id=43f9a6d53a0b43c6ada3d7d2eea1c7d2, scope.project.id=6c9c90447a664a9fa575e9c13ef585f0, user.id=e67cc18cbc114cd3ba4c38bb9ebb91c8) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:00.971262 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b51a4e-0f97-454b-bac7-d268a7f391ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:00.972770 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b51a4e-0f97-454b-bac7-d268a7f391ce None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:00.985658 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 128/251] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:48:00 2022] GET /identity/v3/role_assignments?user.id=e67cc18cbc114cd3ba4c38bb9ebb91c8&scope.project.id=6c9c90447a664a9fa575e9c13ef585f0&role.id=43f9a6d53a0b43c6ada3d7d2eea1c7d2 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:01.021682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff3ecb-e1c4-4025-bffc-3527f9fc9983 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:01.022846 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff3ecb-e1c4-4025-bffc-3527f9fc9983 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:01.022927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff3ecb-e1c4-4025-bffc-3527f9fc9983 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:01.023740 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a94e54-94ab-48e1-b20f-5ca8b95d712a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:01.024670 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a94e54-94ab-48e1-b20f-5ca8b95d712a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:01.024952 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a94e54-94ab-48e1-b20f-5ca8b95d712a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:01.025225 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a94e54-94ab-48e1-b20f-5ca8b95d712a None admin] PATH_INFO: `/v3/projects/6c9c90447a664a9fa575e9c13ef585f0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:01.026268 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 129/252] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:01.027121 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a94e54-94ab-48e1-b20f-5ca8b95d712a None admin] RBAC: Authorizing `identity:get_project(project_id=6c9c90447a664a9fa575e9c13ef585f0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:01.028750 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a94e54-94ab-48e1-b20f-5ca8b95d712a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:01.030572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 124/253] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:48:00 2022] GET /identity/v3/projects/6c9c90447a664a9fa575e9c13ef585f0 => generated 302 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:01.036376 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff3ecb-e1c4-4025-bffc-3527f9fc9983 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:01.053321 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9c2cf-6d8e-4963-97d9-aa4c4d24505c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:01.053661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9c2cf-6d8e-4963-97d9-aa4c4d24505c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:01.053921 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9c2cf-6d8e-4963-97d9-aa4c4d24505c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:01.082851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-98a9c2cf-6d8e-4963-97d9-aa4c4d24505c None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:01.084089 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-98a9c2cf-6d8e-4963-97d9-aa4c4d24505c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:01.092682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:01.199600 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff3ecb-e1c4-4025-bffc-3527f9fc9983 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:01.202972 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 125/254] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:01.204479 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20652d-4c63-4497-a915-7a0b42667d56 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:01.204767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20652d-4c63-4497-a915-7a0b42667d56 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:01.205042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20652d-4c63-4497-a915-7a0b42667d56 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:01.206535 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 126/255] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:01.213916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ffb1f-4e64-4870-be6c-1d896f85327a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:01.214194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ffb1f-4e64-4870-be6c-1d896f85327a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:01.214474 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ffb1f-4e64-4870-be6c-1d896f85327a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:01.244433 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:01.244965 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f0d6e-db96-461b-8e1c-e40670fe87b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:01.245728 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f0d6e-db96-461b-8e1c-e40670fe87b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:01.246017 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f0d6e-db96-461b-8e1c-e40670fe87b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:01.246918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f0d6e-db96-461b-8e1c-e40670fe87b4 None admin] PATH_INFO: `/v3/projects/6c9c90447a664a9fa575e9c13ef585f0/users/9085a6e690aa4aacb85c0f3ff27767dc/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:01.247011 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-dc5ffb1f-4e64-4870-be6c-1d896f85327a None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:01.248398 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-dc5ffb1f-4e64-4870-be6c-1d896f85327a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:01.253209 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f0d6e-db96-461b-8e1c-e40670fe87b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c9c90447a664a9fa575e9c13ef585f0, user_id=9085a6e690aa4aacb85c0f3ff27767dc, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:01.255059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f0d6e-db96-461b-8e1c-e40670fe87b4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:01.339248 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 127/256] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:01.340517 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ffb1f-4e64-4870-be6c-1d896f85327a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:01.380534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 130/257] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jun 23 19:48:01 2022] PUT /identity/v3/projects/6c9c90447a664a9fa575e9c13ef585f0/users/9085a6e690aa4aacb85c0f3ff27767dc/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:48:01.414285 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f0d6e-db96-461b-8e1c-e40670fe87b4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:01.487943 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ffb1f-4e64-4870-be6c-1d896f85327a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:01.526209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3d8f1-e3e7-4e1a-b770-4e44e56594bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:01.527385 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3d8f1-e3e7-4e1a-b770-4e44e56594bc None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:01.527876 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3d8f1-e3e7-4e1a-b770-4e44e56594bc None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:01.528408 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa3d8f1-e3e7-4e1a-b770-4e44e56594bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:01.534484 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:01.547296 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa3d8f1-e3e7-4e1a-b770-4e44e56594bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:01.555648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-3aa3d8f1-e3e7-4e1a-b770-4e44e56594bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:48:01.555648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:01.555648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:01.555648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:01.555648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:01.555648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:01.555648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:01.555648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:01.555648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:01.555648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:01.555648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:01.555648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:01.555648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:01.555648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:01.555648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:01.555648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:01.556767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:01.556767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:01.556767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:01.556767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:01.556767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:01.556767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:01.556767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:01.556767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:01.556767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:01.556767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:01.556767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:01.556767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:01.556767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:01.556767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:01.556767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:01.556767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:01.557563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:01.557563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:01.557563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:01.557563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:01.557563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:01.557563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:01.557563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:01.557563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:01.557563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:01.557563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:01.557563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:48:01.557563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:01.560057 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 128/258] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:48:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:01.577216 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3d8f1-e3e7-4e1a-b770-4e44e56594bc None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:01.650589 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:01.668274 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f0d6e-db96-461b-8e1c-e40670fe87b4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:01.711032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211adc9-ffd7-4f7a-857f-a836dd5d4d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:01.712106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211adc9-ffd7-4f7a-857f-a836dd5d4d2d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:01.712582 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211adc9-ffd7-4f7a-857f-a836dd5d4d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:01.713110 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211adc9-ffd7-4f7a-857f-a836dd5d4d2d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:01.723898 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211adc9-ffd7-4f7a-857f-a836dd5d4d2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:01.735941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-b211adc9-ffd7-4f7a-857f-a836dd5d4d2d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 19:48:01.735941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:01.735941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:01.735941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:01.735941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:01.735941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:01.735941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:01.735941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:01.735941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:01.735941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:01.735941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:01.735941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:01.735941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:01.735941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:01.735941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:01.735941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:01.737132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:01.737132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:01.737132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:01.737132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:01.737132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:01.737132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:01.737132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:01.737132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:01.737132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:01.737132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:01.737132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:01.737132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:01.737132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:01.737132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:01.737132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:01.737132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:01.738145 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:01.738145 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:01.738145 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:01.738145 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:01.738145 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:01.738145 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:01.738145 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:01.738145 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:01.738145 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:01.738145 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:01.738145 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 19:48:01.738145 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:01.740462 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 131/259] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jun 23 19:48:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:48:01.746699 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211adc9-ffd7-4f7a-857f-a836dd5d4d2d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:01.816963 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa3d8f1-e3e7-4e1a-b770-4e44e56594bc None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:01.856319 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3ce82-8ae3-4e4d-bcf3-1836fe665c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:01.857622 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3ce82-8ae3-4e4d-bcf3-1836fe665c47 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:01.858042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3ce82-8ae3-4e4d-bcf3-1836fe665c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:01.858485 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3ce82-8ae3-4e4d-bcf3-1836fe665c47 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:01.859372 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee3ce82-8ae3-4e4d-bcf3-1836fe665c47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:01.859816 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee3ce82-8ae3-4e4d-bcf3-1836fe665c47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:01.861537 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee3ce82-8ae3-4e4d-bcf3-1836fe665c47 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:01.871189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 129/260] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:01.879697 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3ce82-8ae3-4e4d-bcf3-1836fe665c47 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:01.949111 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:01.979098 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211adc9-ffd7-4f7a-857f-a836dd5d4d2d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:01.997749 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3ce82-8ae3-4e4d-bcf3-1836fe665c47 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:02.036214 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322f815-ea8f-4145-b004-a001ddf3c09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:02.037432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322f815-ea8f-4145-b004-a001ddf3c09c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:02.038432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322f815-ea8f-4145-b004-a001ddf3c09c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:02.038849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322f815-ea8f-4145-b004-a001ddf3c09c None admin] PATH_INFO: `/v3/users/ff8ba25ac61f4edf86d1507803104750` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:02.040289 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322f815-ea8f-4145-b004-a001ddf3c09c None admin] RBAC: Authorizing `identity:get_user(user_id=ff8ba25ac61f4edf86d1507803104750)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:02.041943 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322f815-ea8f-4145-b004-a001ddf3c09c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:02.044564 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 130/261] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:48:01 2022] GET /identity/v3/users/ff8ba25ac61f4edf86d1507803104750 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:02.052538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322f815-ea8f-4145-b004-a001ddf3c09c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:02.075900 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a11c4b2-e5a4-442d-9a1e-e213fc548753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:02.076612 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a11c4b2-e5a4-442d-9a1e-e213fc548753 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:02.076895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a11c4b2-e5a4-442d-9a1e-e213fc548753 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:02.077170 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a11c4b2-e5a4-442d-9a1e-e213fc548753 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:02.077933 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a11c4b2-e5a4-442d-9a1e-e213fc548753 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:02.078240 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a11c4b2-e5a4-442d-9a1e-e213fc548753 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:02.079626 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a11c4b2-e5a4-442d-9a1e-e213fc548753 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:02.098333 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 132/262] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:48:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:02.103286 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a11c4b2-e5a4-442d-9a1e-e213fc548753 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:02.122196 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:02.271446 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322f815-ea8f-4145-b004-a001ddf3c09c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:02.317587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:02.321700 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a11c4b2-e5a4-442d-9a1e-e213fc548753 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:02.336776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b5817-fabe-41d1-ac28-79118f7a9cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:02.337436 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b5817-fabe-41d1-ac28-79118f7a9cdd None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:02.337723 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b5817-fabe-41d1-ac28-79118f7a9cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:02.338002 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b5817-fabe-41d1-ac28-79118f7a9cdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:02.389625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb202d-e18d-471e-8005-aa18257df443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:02.390555 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb202d-e18d-471e-8005-aa18257df443 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:02.390838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb202d-e18d-471e-8005-aa18257df443 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:02.391109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb202d-e18d-471e-8005-aa18257df443 None admin] PATH_INFO: `/v3/users/ff8ba25ac61f4edf86d1507803104750` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:02.392328 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb202d-e18d-471e-8005-aa18257df443 None admin] RBAC: Authorizing `identity:get_user(user_id=ff8ba25ac61f4edf86d1507803104750)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:02.394009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb202d-e18d-471e-8005-aa18257df443 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:02.400747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 133/263] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:48:02 2022] GET /identity/v3/users/ff8ba25ac61f4edf86d1507803104750 => generated 283 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:02.401758 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb202d-e18d-471e-8005-aa18257df443 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:02.466831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b5817-fabe-41d1-ac28-79118f7a9cdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:02.468555 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b5817-fabe-41d1-ac28-79118f7a9cdd None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:02.540926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:02.594085 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb202d-e18d-471e-8005-aa18257df443 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:02.653564 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93a5a9-a7a7-410b-8437-bf85b4e66844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:02.654251 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93a5a9-a7a7-410b-8437-bf85b4e66844 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:02.654505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93a5a9-a7a7-410b-8437-bf85b4e66844 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:02.654788 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93a5a9-a7a7-410b-8437-bf85b4e66844 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:02.687659 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 131/264] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:48:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 635 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:48:02.717810 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b5817-fabe-41d1-ac28-79118f7a9cdd None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:02.752008 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93a5a9-a7a7-410b-8437-bf85b4e66844 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:02.753553 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93a5a9-a7a7-410b-8437-bf85b4e66844 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:02.793152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:02.906133 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b5817-fabe-41d1-ac28-79118f7a9cdd None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:02.916653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:02.955698 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4636cb-adb8-442f-abcc-6c93146898fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:02.956594 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4636cb-adb8-442f-abcc-6c93146898fd None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:02.956876 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4636cb-adb8-442f-abcc-6c93146898fd None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:02.957148 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4636cb-adb8-442f-abcc-6c93146898fd None admin] PATH_INFO: `/v3/projects/6c9c90447a664a9fa575e9c13ef585f0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:02.958342 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4636cb-adb8-442f-abcc-6c93146898fd None admin] RBAC: Authorizing `identity:get_project(project_id=6c9c90447a664a9fa575e9c13ef585f0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:02.960444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4636cb-adb8-442f-abcc-6c93146898fd None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:02.962458 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 132/265] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:48:02 2022] GET /identity/v3/projects/6c9c90447a664a9fa575e9c13ef585f0 => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:02.968522 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4636cb-adb8-442f-abcc-6c93146898fd None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:02.984632 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 134/266] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:48:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:48:02.990259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93a5a9-a7a7-410b-8437-bf85b4e66844 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:03.124963 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:03.131021 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4636cb-adb8-442f-abcc-6c93146898fd None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:03.190071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1544e15-c084-4e4e-b136-196684fa990a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:03.190737 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1544e15-c084-4e4e-b136-196684fa990a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:03.191018 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1544e15-c084-4e4e-b136-196684fa990a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:03.191291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1544e15-c084-4e4e-b136-196684fa990a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:03.192122 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1544e15-c084-4e4e-b136-196684fa990a None admin] RBAC: Adding query filter params (role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb, scope.project.id=6c9c90447a664a9fa575e9c13ef585f0, user.id=ff8ba25ac61f4edf86d1507803104750) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:03.192458 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1544e15-c084-4e4e-b136-196684fa990a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:03.193915 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1544e15-c084-4e4e-b136-196684fa990a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:03.197024 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93a5a9-a7a7-410b-8437-bf85b4e66844 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:03.205165 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 133/267] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:48:02 2022] GET /identity/v3/role_assignments?user.id=ff8ba25ac61f4edf86d1507803104750&scope.project.id=6c9c90447a664a9fa575e9c13ef585f0&role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:03.258401 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4571a-9e63-4e23-aa19-baa61d1c3bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:03.259292 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4571a-9e63-4e23-aa19-baa61d1c3bcc None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:03.259578 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4571a-9e63-4e23-aa19-baa61d1c3bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:03.259854 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4571a-9e63-4e23-aa19-baa61d1c3bcc None admin] PATH_INFO: `/v3/projects/6c9c90447a664a9fa575e9c13ef585f0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:03.262430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4571a-9e63-4e23-aa19-baa61d1c3bcc None admin] RBAC: Authorizing `identity:get_project(project_id=6c9c90447a664a9fa575e9c13ef585f0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:03.263914 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4571a-9e63-4e23-aa19-baa61d1c3bcc None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:03.266043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 135/268] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:48:02 2022] GET /identity/v3/projects/6c9c90447a664a9fa575e9c13ef585f0 => generated 302 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:03.274215 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1544e15-c084-4e4e-b136-196684fa990a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:03.381140 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf7b2e-9993-4b13-9358-0a85c23c7152 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:03.381433 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf7b2e-9993-4b13-9358-0a85c23c7152 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:03.382236 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf7b2e-9993-4b13-9358-0a85c23c7152 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:03.386069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 136/269] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:03.391049 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c466e3-22bc-454a-bbbd-18169d06eff7 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:03.391665 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c466e3-22bc-454a-bbbd-18169d06eff7 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:03.391665 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c466e3-22bc-454a-bbbd-18169d06eff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:03.395973 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1544e15-c084-4e4e-b136-196684fa990a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:03.422825 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a8c466e3-22bc-454a-bbbd-18169d06eff7 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:03.424232 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a8c466e3-22bc-454a-bbbd-18169d06eff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:03.442055 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82314f6-c012-45cd-821e-5c810aa4a59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:03.446447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82314f6-c012-45cd-821e-5c810aa4a59a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:03.446447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82314f6-c012-45cd-821e-5c810aa4a59a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:03.446447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82314f6-c012-45cd-821e-5c810aa4a59a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:03.446447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82314f6-c012-45cd-821e-5c810aa4a59a None admin] RBAC: Adding query filter params (role.id=03e4cfc753754643ae08fa909647cfe3, scope.project.id=6c9c90447a664a9fa575e9c13ef585f0, user.id=ff8ba25ac61f4edf86d1507803104750) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:03.446447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82314f6-c012-45cd-821e-5c810aa4a59a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:03.447524 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82314f6-c012-45cd-821e-5c810aa4a59a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:03.461755 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 134/270] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:48:03 2022] GET /identity/v3/role_assignments?user.id=ff8ba25ac61f4edf86d1507803104750&scope.project.id=6c9c90447a664a9fa575e9c13ef585f0&role.id=03e4cfc753754643ae08fa909647cfe3 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:03.480729 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:03.511480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594f313-4f9f-4b5e-850b-64a4313beaba None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:03.512126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594f313-4f9f-4b5e-850b-64a4313beaba None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:03.512683 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594f313-4f9f-4b5e-850b-64a4313beaba None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:03.514268 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 135/271] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:48:03.520085 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655373f-8bbc-4b92-807c-0ebaa53bcadb None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:03.520651 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655373f-8bbc-4b92-807c-0ebaa53bcadb None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:03.521189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655373f-8bbc-4b92-807c-0ebaa53bcadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:03.535276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 137/272] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:03.539836 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3655373f-8bbc-4b92-807c-0ebaa53bcadb None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:03.541361 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3655373f-8bbc-4b92-807c-0ebaa53bcadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:03.569992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:03.576237 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c466e3-22bc-454a-bbbd-18169d06eff7 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:03.638720 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 136/273] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:03.686226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655373f-8bbc-4b92-807c-0ebaa53bcadb None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:03.786307 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c466e3-22bc-454a-bbbd-18169d06eff7 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:03.822059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655373f-8bbc-4b92-807c-0ebaa53bcadb None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:03.832364 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c85b8d-9e1a-4af1-91a5-8a45ea92115c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:03.833190 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c85b8d-9e1a-4af1-91a5-8a45ea92115c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:03.833474 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c85b8d-9e1a-4af1-91a5-8a45ea92115c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:03.834124 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c85b8d-9e1a-4af1-91a5-8a45ea92115c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:03.849324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c85b8d-9e1a-4af1-91a5-8a45ea92115c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:03.859245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-27c85b8d-9e1a-4af1-91a5-8a45ea92115c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:48:03.859245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:03.859245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:03.859245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:03.859245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:03.859245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:03.859245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:03.859245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:03.859245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:03.859245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:03.859245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:03.859245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:03.859245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:03.859245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:03.859245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:03.859245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:03.860175 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:03.860175 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:03.860175 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:03.860175 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:03.860175 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:03.860175 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:03.860175 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:03.860175 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:03.860175 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:03.860175 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:03.860175 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:03.860175 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:03.860175 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:03.860175 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:03.860175 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:03.860175 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:03.861071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:03.861071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:03.861071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:03.861071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:03.861071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:03.861071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:03.861071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:03.861071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:03.861071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:03.861071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:03.861071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:48:03.861071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:03.861071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 138/274] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:03.862051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ab79d-0389-46e6-8fbc-82950edf35c8 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:03.862051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ab79d-0389-46e6-8fbc-82950edf35c8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:03.862347 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ab79d-0389-46e6-8fbc-82950edf35c8 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:03.863656 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 139/275] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:03.866746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ab79d-0389-46e6-8fbc-82950edf35c8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:03.870043 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61cbe42-eb9b-4523-8ec5-db1d9e447fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:03.870908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61cbe42-eb9b-4523-8ec5-db1d9e447fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:03.871172 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61cbe42-eb9b-4523-8ec5-db1d9e447fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:03.871459 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61cbe42-eb9b-4523-8ec5-db1d9e447fe6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:03.872713 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61cbe42-eb9b-4523-8ec5-db1d9e447fe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:03.874193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61cbe42-eb9b-4523-8ec5-db1d9e447fe6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:03.876270 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 137/276] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:03.877479 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbc98f-e719-4f41-a492-0ee114d101b9 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:03.877813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbc98f-e719-4f41-a492-0ee114d101b9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:03.878091 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbc98f-e719-4f41-a492-0ee114d101b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:03.894998 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-07cbc98f-e719-4f41-a492-0ee114d101b9 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:03.896120 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-07cbc98f-e719-4f41-a492-0ee114d101b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:03.903251 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:03.911489 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:03.968838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 138/277] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:03.968838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbc98f-e719-4f41-a492-0ee114d101b9 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:03.986669 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ab79d-0389-46e6-8fbc-82950edf35c8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:04.023945 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0133f5db-a444-467f-a3e5-18ca41fbe752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:04.024640 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133f5db-a444-467f-a3e5-18ca41fbe752 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:04.024922 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133f5db-a444-467f-a3e5-18ca41fbe752 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:04.025195 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0133f5db-a444-467f-a3e5-18ca41fbe752 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:04.025943 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0133f5db-a444-467f-a3e5-18ca41fbe752 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:04.026249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0133f5db-a444-467f-a3e5-18ca41fbe752 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:04.027623 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0133f5db-a444-467f-a3e5-18ca41fbe752 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:04.036701 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 140/278] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jun 23 19:48:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:04.041990 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0133f5db-a444-467f-a3e5-18ca41fbe752 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:04.070649 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbc98f-e719-4f41-a492-0ee114d101b9 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:04.105396 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccbd0cf-6ba8-4a37-acfe-9c9ba4e7886c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:04.106060 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbd0cf-6ba8-4a37-acfe-9c9ba4e7886c None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:04.106340 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbd0cf-6ba8-4a37-acfe-9c9ba4e7886c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:04.106612 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbd0cf-6ba8-4a37-acfe-9c9ba4e7886c None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:04.107310 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccbd0cf-6ba8-4a37-acfe-9c9ba4e7886c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:04.108830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccbd0cf-6ba8-4a37-acfe-9c9ba4e7886c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:04.145519 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:04.152755 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0133f5db-a444-467f-a3e5-18ca41fbe752 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:04.186610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd65ab5-c348-4e59-a565-4860f21dbdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:04.187431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd65ab5-c348-4e59-a565-4860f21dbdd3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:04.188015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd65ab5-c348-4e59-a565-4860f21dbdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:04.188289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd65ab5-c348-4e59-a565-4860f21dbdd3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:04.198808 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd65ab5-c348-4e59-a565-4860f21dbdd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:04.206629 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-1dd65ab5-c348-4e59-a565-4860f21dbdd3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 19:48:04.206629 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:04.206629 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:04.206629 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:04.206629 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:04.206629 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:04.206629 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:04.206629 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:04.206629 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:04.206629 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:04.206629 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:04.206629 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:04.206629 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:04.206629 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:04.206629 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:04.206629 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:04.207692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:04.207692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:04.207692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:04.207692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:04.207692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:04.207692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:04.207692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:04.207692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:04.207692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:04.207692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:04.207692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:04.207692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:04.207692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:04.207692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:04.207692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:04.207692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:04.208515 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:04.208515 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:04.208515 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:04.208515 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:04.208515 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:04.208515 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:04.208515 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:04.208515 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:04.208515 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:04.208515 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:04.208515 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 19:48:04.208515 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:04.208515 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 141/279] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:04.213409 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd65ab5-c348-4e59-a565-4860f21dbdd3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:04.217641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 139/280] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jun 23 19:48:03 2022] POST /identity/v3/users => generated 293 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:48:04.217848 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccbd0cf-6ba8-4a37-acfe-9c9ba4e7886c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:04.235798 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:04.346852 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccbd0cf-6ba8-4a37-acfe-9c9ba4e7886c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:04.356201 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd65ab5-c348-4e59-a565-4860f21dbdd3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:04.385876 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a64fc-59a7-482b-8b81-564a723b27c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:04.387029 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a64fc-59a7-482b-8b81-564a723b27c6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:04.387029 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a64fc-59a7-482b-8b81-564a723b27c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:04.387194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a64fc-59a7-482b-8b81-564a723b27c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:04.387944 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a64fc-59a7-482b-8b81-564a723b27c6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:04.388253 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a64fc-59a7-482b-8b81-564a723b27c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:04.389752 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a64fc-59a7-482b-8b81-564a723b27c6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:04.399342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023e802-aeb1-40f3-ac4b-be73810a08fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:04.399342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023e802-aeb1-40f3-ac4b-be73810a08fe None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:04.399582 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 140/281] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jun 23 19:48:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:04.399665 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023e802-aeb1-40f3-ac4b-be73810a08fe None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:04.399665 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023e802-aeb1-40f3-ac4b-be73810a08fe None admin] PATH_INFO: `/v3/users/a9168a7cac5a4622add75e1fb21c4ef7` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:04.400905 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c53289-3137-4882-997d-6bccc229a483 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:04.401196 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c53289-3137-4882-997d-6bccc229a483 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:04.401505 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c53289-3137-4882-997d-6bccc229a483 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:04.402867 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 141/282] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:04.405387 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c53289-3137-4882-997d-6bccc229a483 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:04.414336 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6023e802-aeb1-40f3-ac4b-be73810a08fe None admin] RBAC: Authorizing `identity:get_user(user_id=a9168a7cac5a4622add75e1fb21c4ef7)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:04.415830 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6023e802-aeb1-40f3-ac4b-be73810a08fe None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:04.417663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 142/283] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:48:04 2022] GET /identity/v3/users/a9168a7cac5a4622add75e1fb21c4ef7 => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:04.418927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8dfbf-6f44-4936-9f33-b62c69031b55 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:04.419211 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8dfbf-6f44-4936-9f33-b62c69031b55 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:04.419512 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8dfbf-6f44-4936-9f33-b62c69031b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:04.438620 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-93e8dfbf-6f44-4936-9f33-b62c69031b55 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:04.439945 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-93e8dfbf-6f44-4936-9f33-b62c69031b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:04.449894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:04.495667 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:04.532718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 143/284] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:04.533589 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8dfbf-6f44-4936-9f33-b62c69031b55 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:04.537406 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c53289-3137-4882-997d-6bccc229a483 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:04.577431 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92ec54-2e41-4f9d-939e-21b417a7a373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:04.578696 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92ec54-2e41-4f9d-939e-21b417a7a373 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:04.579182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92ec54-2e41-4f9d-939e-21b417a7a373 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:04.579554 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92ec54-2e41-4f9d-939e-21b417a7a373 None admin] PATH_INFO: `/v3/users/e67cc18cbc114cd3ba4c38bb9ebb91c8` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:04.580803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92ec54-2e41-4f9d-939e-21b417a7a373 None admin] RBAC: Authorizing `identity:get_user(user_id=e67cc18cbc114cd3ba4c38bb9ebb91c8)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:04.583705 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92ec54-2e41-4f9d-939e-21b417a7a373 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:04.585729 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 142/285] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:48:04 2022] GET /identity/v3/users/e67cc18cbc114cd3ba4c38bb9ebb91c8 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:04.586503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92ec54-2e41-4f9d-939e-21b417a7a373 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:04.645211 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8dfbf-6f44-4936-9f33-b62c69031b55 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:04.680612 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530106e-a6df-4274-aa7b-a1bb3b0a22d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:04.681239 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530106e-a6df-4274-aa7b-a1bb3b0a22d1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:04.681521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530106e-a6df-4274-aa7b-a1bb3b0a22d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:04.681797 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530106e-a6df-4274-aa7b-a1bb3b0a22d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:04.682681 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3530106e-a6df-4274-aa7b-a1bb3b0a22d1 None admin] RBAC: Adding query filter params (role.id=860491bfbb574772a7a213d6a25962e0, user.id=a9168a7cac5a4622add75e1fb21c4ef7, scope.system=all) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:04.683005 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3530106e-a6df-4274-aa7b-a1bb3b0a22d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:04.684499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3530106e-a6df-4274-aa7b-a1bb3b0a22d1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:04.690433 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 144/286] 10.222.0.44 () {56 vars in 1308 bytes} [Thu Jun 23 19:48:04 2022] GET /identity/v3/role_assignments?user.id=a9168a7cac5a4622add75e1fb21c4ef7&scope.system=all&role.id=860491bfbb574772a7a213d6a25962e0 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:04.691455 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530106e-a6df-4274-aa7b-a1bb3b0a22d1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:04.700607 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:04.721364 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92ec54-2e41-4f9d-939e-21b417a7a373 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:04.768516 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dde00a-7cdb-43bf-a1e7-c67ebfcd9413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:04.769706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dde00a-7cdb-43bf-a1e7-c67ebfcd9413 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:04.770185 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dde00a-7cdb-43bf-a1e7-c67ebfcd9413 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:04.770825 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dde00a-7cdb-43bf-a1e7-c67ebfcd9413 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:04.784734 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:04.795205 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dde00a-7cdb-43bf-a1e7-c67ebfcd9413 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:04.804572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-d6dde00a-7cdb-43bf-a1e7-c67ebfcd9413 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:48:04.804572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:04.804572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:04.804572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:04.804572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:04.804572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:04.804572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:04.804572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:04.804572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:04.804572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:04.804572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:04.804572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:04.804572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:04.804572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:04.804572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:04.804572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:04.814856 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:04.814856 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:04.814856 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:04.814856 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:04.814856 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:04.814856 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:04.814856 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:04.814856 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:04.814856 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:04.814856 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:04.814856 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:04.814856 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:04.814856 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:04.814856 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:04.814856 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:04.814856 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:04.815657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:04.815657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:04.815657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:04.815657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:04.815657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:04.815657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:04.815657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:04.815657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:04.815657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:04.815657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:04.815657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:48:04.815657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:04.815657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 143/287] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:04.817822 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dde00a-7cdb-43bf-a1e7-c67ebfcd9413 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:04.869442 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530106e-a6df-4274-aa7b-a1bb3b0a22d1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:04.932885 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d50dfa-b582-4c0c-9c27-068cedaf2276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:04.934032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d50dfa-b582-4c0c-9c27-068cedaf2276 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:04.934142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d50dfa-b582-4c0c-9c27-068cedaf2276 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:04.934352 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d50dfa-b582-4c0c-9c27-068cedaf2276 None admin] PATH_INFO: `/v3/projects/6c9c90447a664a9fa575e9c13ef585f0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:04.935596 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d50dfa-b582-4c0c-9c27-068cedaf2276 None admin] RBAC: Authorizing `identity:get_project(project_id=6c9c90447a664a9fa575e9c13ef585f0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:04.937092 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d50dfa-b582-4c0c-9c27-068cedaf2276 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:04.944665 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 145/288] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:48:04 2022] GET /identity/v3/projects/6c9c90447a664a9fa575e9c13ef585f0 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:04.945370 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d50dfa-b582-4c0c-9c27-068cedaf2276 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:04.973059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dde00a-7cdb-43bf-a1e7-c67ebfcd9413 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:05.012566 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a9e12-512f-4119-9784-41b132b75f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:05.014108 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a9e12-512f-4119-9784-41b132b75f88 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:05.014585 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a9e12-512f-4119-9784-41b132b75f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:05.015048 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a9e12-512f-4119-9784-41b132b75f88 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:05.015979 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2a9e12-512f-4119-9784-41b132b75f88 None admin] RBAC: Adding query filter params (name=member) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:05.016647 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2a9e12-512f-4119-9784-41b132b75f88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:05.018239 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2a9e12-512f-4119-9784-41b132b75f88 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:05.028134 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 144/289] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jun 23 19:48:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:05.044347 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a9e12-512f-4119-9784-41b132b75f88 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:05.098031 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:05.151054 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d50dfa-b582-4c0c-9c27-068cedaf2276 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:05.166969 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:05.198544 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b004516-45c2-4723-a253-539359fd0ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:05.199358 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b004516-45c2-4723-a253-539359fd0ebf None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:05.199900 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b004516-45c2-4723-a253-539359fd0ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:05.199900 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b004516-45c2-4723-a253-539359fd0ebf None admin] PATH_INFO: `/v3/projects/6c9c90447a664a9fa575e9c13ef585f0/users/e67cc18cbc114cd3ba4c38bb9ebb91c8/roles/43f9a6d53a0b43c6ada3d7d2eea1c7d2` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:05.203534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b004516-45c2-4723-a253-539359fd0ebf None admin] RBAC: Authorizing `identity:create_grant(project_id=6c9c90447a664a9fa575e9c13ef585f0, user_id=e67cc18cbc114cd3ba4c38bb9ebb91c8, role_id=43f9a6d53a0b43c6ada3d7d2eea1c7d2)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:05.205036 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b004516-45c2-4723-a253-539359fd0ebf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:05.233701 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2a9e12-512f-4119-9784-41b132b75f88 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:05.269447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7376ff-2d39-442a-9d70-cd98eb5cf754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:05.270323 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7376ff-2d39-442a-9d70-cd98eb5cf754 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:05.270608 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7376ff-2d39-442a-9d70-cd98eb5cf754 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:05.270883 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7376ff-2d39-442a-9d70-cd98eb5cf754 None admin] PATH_INFO: `/v3/users/9085a6e690aa4aacb85c0f3ff27767dc` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:05.272199 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7376ff-2d39-442a-9d70-cd98eb5cf754 None admin] RBAC: Authorizing `identity:get_user(user_id=9085a6e690aa4aacb85c0f3ff27767dc)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:05.273753 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7376ff-2d39-442a-9d70-cd98eb5cf754 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:05.280318 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 145/290] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:48:05 2022] GET /identity/v3/users/9085a6e690aa4aacb85c0f3ff27767dc => generated 297 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:05.287409 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 146/291] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jun 23 19:48:04 2022] PUT /identity/v3/projects/6c9c90447a664a9fa575e9c13ef585f0/users/e67cc18cbc114cd3ba4c38bb9ebb91c8/roles/43f9a6d53a0b43c6ada3d7d2eea1c7d2 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:48:05.292013 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7376ff-2d39-442a-9d70-cd98eb5cf754 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:05.399102 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7376ff-2d39-442a-9d70-cd98eb5cf754 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:05.427902 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:05.437194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0bc07e-4383-4f10-866b-beaa7e9d8efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:05.438125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bc07e-4383-4f10-866b-beaa7e9d8efa None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:05.438407 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bc07e-4383-4f10-866b-beaa7e9d8efa None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:05.438683 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bc07e-4383-4f10-866b-beaa7e9d8efa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:05.510574 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0bc07e-4383-4f10-866b-beaa7e9d8efa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:05.511984 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0bc07e-4383-4f10-866b-beaa7e9d8efa None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:05.636506 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 146/292] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:48:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:48:05.637863 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b004516-45c2-4723-a253-539359fd0ebf None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:05.752997 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b004516-45c2-4723-a253-539359fd0ebf None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:05.799205 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c9973-ce2f-4fb4-8dda-2aa49e332e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:05.799999 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c9973-ce2f-4fb4-8dda-2aa49e332e6a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:05.800319 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c9973-ce2f-4fb4-8dda-2aa49e332e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:05.800592 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c9973-ce2f-4fb4-8dda-2aa49e332e6a None admin] PATH_INFO: `/v3/projects/6c9c90447a664a9fa575e9c13ef585f0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:05.805031 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c9973-ce2f-4fb4-8dda-2aa49e332e6a None admin] RBAC: Authorizing `identity:get_project(project_id=6c9c90447a664a9fa575e9c13ef585f0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:05.806546 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c9973-ce2f-4fb4-8dda-2aa49e332e6a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:05.808258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 147/293] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:48:05 2022] GET /identity/v3/projects/6c9c90447a664a9fa575e9c13ef585f0 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:05.814545 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0bc07e-4383-4f10-866b-beaa7e9d8efa None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:05.855762 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:05.953797 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0bc07e-4383-4f10-866b-beaa7e9d8efa None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:06.004105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015a13c-6b16-42ed-9789-d799a362b8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:06.004796 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015a13c-6b16-42ed-9789-d799a362b8d1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:06.005056 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015a13c-6b16-42ed-9789-d799a362b8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:06.005334 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015a13c-6b16-42ed-9789-d799a362b8d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:06.006168 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015a13c-6b16-42ed-9789-d799a362b8d1 None admin] RBAC: Adding query filter params (role.id=860491bfbb574772a7a213d6a25962e0, scope.project.id=6c9c90447a664a9fa575e9c13ef585f0, user.id=9085a6e690aa4aacb85c0f3ff27767dc) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:06.007023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015a13c-6b16-42ed-9789-d799a362b8d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:06.008507 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015a13c-6b16-42ed-9789-d799a362b8d1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:06.014779 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 147/294] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:48:05 2022] GET /identity/v3/role_assignments?user.id=9085a6e690aa4aacb85c0f3ff27767dc&scope.project.id=6c9c90447a664a9fa575e9c13ef585f0&role.id=860491bfbb574772a7a213d6a25962e0 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:06.611333 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220a5ff-64e9-40b4-bad4-1247997dde19 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:06.611859 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220a5ff-64e9-40b4-bad4-1247997dde19 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:06.611859 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220a5ff-64e9-40b4-bad4-1247997dde19 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:06.613558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 148/295] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:06.619312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b96602-20a0-4c36-a14a-673cc4d8cf84 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:06.619607 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b96602-20a0-4c36-a14a-673cc4d8cf84 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:06.619881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b96602-20a0-4c36-a14a-673cc4d8cf84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:06.637138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-42b96602-20a0-4c36-a14a-673cc4d8cf84 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:06.638343 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-42b96602-20a0-4c36-a14a-673cc4d8cf84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:06.714013 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 148/296] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:06.738699 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb98e6-df90-4f9d-bbba-ba88072c1718 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:06.739011 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb98e6-df90-4f9d-bbba-ba88072c1718 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:06.739261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb98e6-df90-4f9d-bbba-ba88072c1718 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:06.740889 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 149/297] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:06.741851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b96602-20a0-4c36-a14a-673cc4d8cf84 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:06.745649 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8202ede-fb4c-4d52-8243-33a9b7abcf98 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:06.745947 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8202ede-fb4c-4d52-8243-33a9b7abcf98 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:06.746233 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8202ede-fb4c-4d52-8243-33a9b7abcf98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:06.751281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:06.765292 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e8202ede-fb4c-4d52-8243-33a9b7abcf98 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:06.769638 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e8202ede-fb4c-4d52-8243-33a9b7abcf98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:06.770870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:06.835499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 150/298] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 19:48:06.859714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8202ede-fb4c-4d52-8243-33a9b7abcf98 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:06.863809 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b96602-20a0-4c36-a14a-673cc4d8cf84 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:06.906512 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9042d23-f897-4f0c-94bc-0d7feeac4b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:06.907717 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9042d23-f897-4f0c-94bc-0d7feeac4b6e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:06.908780 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9042d23-f897-4f0c-94bc-0d7feeac4b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:06.909065 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9042d23-f897-4f0c-94bc-0d7feeac4b6e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:06.912336 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9042d23-f897-4f0c-94bc-0d7feeac4b6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:06.914392 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9042d23-f897-4f0c-94bc-0d7feeac4b6e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:06.916123 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 149/299] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:06.921641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9042d23-f897-4f0c-94bc-0d7feeac4b6e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:06.983917 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8202ede-fb4c-4d52-8243-33a9b7abcf98 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:07.016437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:07.022411 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3e2aa-62b3-48e5-9f78-897c71da2cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:07.030586 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3e2aa-62b3-48e5-9f78-897c71da2cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:07.030586 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3e2aa-62b3-48e5-9f78-897c71da2cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:07.030586 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3e2aa-62b3-48e5-9f78-897c71da2cc2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:07.034295 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc3e2aa-62b3-48e5-9f78-897c71da2cc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:07.039063 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:07.042559 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-9fc3e2aa-62b3-48e5-9f78-897c71da2cc2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 19:48:07.042559 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:07.042559 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:07.042559 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:07.042559 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:07.042559 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:07.042559 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:07.042559 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:07.042559 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:07.042559 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:07.042559 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:07.042559 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:07.042559 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:07.042559 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:07.042559 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:07.042559 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:07.043457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:07.043457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:07.043457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:07.043457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:07.043457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:07.043457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:07.043457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:07.043457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:07.043457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:07.043457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:07.043457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:07.043457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:07.043457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:07.043457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:07.043457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:07.043457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:07.044278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:07.044278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:07.044278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:07.044278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:07.044278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:07.044278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:07.044278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:07.044278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:07.044278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:07.044278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:07.044278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 19:48:07.044278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:07.048544 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 151/300] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:48:07.049835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3e2aa-62b3-48e5-9f78-897c71da2cc2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:07.056916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9042d23-f897-4f0c-94bc-0d7feeac4b6e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:07.094624 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1647cca-1040-4f70-a2a7-121404d96984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:07.095291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1647cca-1040-4f70-a2a7-121404d96984 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:07.095579 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1647cca-1040-4f70-a2a7-121404d96984 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:07.095850 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1647cca-1040-4f70-a2a7-121404d96984 None admin] PATH_INFO: `/v3/groups` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:07.096587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1647cca-1040-4f70-a2a7-121404d96984 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:07.097997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1647cca-1040-4f70-a2a7-121404d96984 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:07.151394 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3e2aa-62b3-48e5-9f78-897c71da2cc2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:07.174003 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 150/301] 10.222.0.44 () {60 vars in 1134 bytes} [Thu Jun 23 19:48:06 2022] POST /identity/v3/groups => generated 229 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:48:07.175329 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73404bd-faeb-4f5b-8328-90e980fcb0d3 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:07.175690 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73404bd-faeb-4f5b-8328-90e980fcb0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:07.176047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73404bd-faeb-4f5b-8328-90e980fcb0d3 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:07.182445 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 151/302] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:07.184038 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb646f-a32f-4dd8-8e59-fb52d79405f8 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:07.184481 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb646f-a32f-4dd8-8e59-fb52d79405f8 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:07.184916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb646f-a32f-4dd8-8e59-fb52d79405f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:07.190164 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cec4fb-93f2-4826-9577-34d26953e6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:07.191559 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cec4fb-93f2-4826-9577-34d26953e6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:07.192025 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cec4fb-93f2-4826-9577-34d26953e6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:07.193527 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cec4fb-93f2-4826-9577-34d26953e6d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:07.194688 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cec4fb-93f2-4826-9577-34d26953e6d8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:07.195201 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cec4fb-93f2-4826-9577-34d26953e6d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:07.197028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cec4fb-93f2-4826-9577-34d26953e6d8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:07.206488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-eedb646f-a32f-4dd8-8e59-fb52d79405f8 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:07.207849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-eedb646f-a32f-4dd8-8e59-fb52d79405f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:07.209197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 152/303] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jun 23 19:48:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:07.215318 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cec4fb-93f2-4826-9577-34d26953e6d8 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:07.293643 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:07.295490 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 152/304] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:07.321185 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb646f-a32f-4dd8-8e59-fb52d79405f8 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:07.354286 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cec4fb-93f2-4826-9577-34d26953e6d8 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:07.399819 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21879ed3-d405-4a35-a5fb-8f6e42e0354c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:07.400853 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21879ed3-d405-4a35-a5fb-8f6e42e0354c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:07.401134 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21879ed3-d405-4a35-a5fb-8f6e42e0354c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:07.401456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21879ed3-d405-4a35-a5fb-8f6e42e0354c None admin] PATH_INFO: `/v3/users/51d15ee3ffd24d7898ecc729628d7e96` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:07.421519 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:07.422043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21879ed3-d405-4a35-a5fb-8f6e42e0354c None admin] RBAC: Authorizing `identity:get_user(user_id=51d15ee3ffd24d7898ecc729628d7e96)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:07.424069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21879ed3-d405-4a35-a5fb-8f6e42e0354c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:07.426114 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 153/305] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:48:07 2022] GET /identity/v3/users/51d15ee3ffd24d7898ecc729628d7e96 => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:07.432783 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21879ed3-d405-4a35-a5fb-8f6e42e0354c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:07.445371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb646f-a32f-4dd8-8e59-fb52d79405f8 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:07.480718 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac33e7c-d9ed-4f0d-871b-754eeaa774cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:07.481582 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac33e7c-d9ed-4f0d-871b-754eeaa774cc None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:07.481959 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac33e7c-d9ed-4f0d-871b-754eeaa774cc None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:07.482309 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac33e7c-d9ed-4f0d-871b-754eeaa774cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:07.493495 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac33e7c-d9ed-4f0d-871b-754eeaa774cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:07.502069 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-0ac33e7c-d9ed-4f0d-871b-754eeaa774cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:48:07.502069 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:07.502069 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:07.502069 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:07.502069 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:07.502069 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:07.502069 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:07.502069 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:07.502069 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:07.502069 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:07.502069 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:07.502069 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:07.502069 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:07.502069 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:07.502069 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:07.502069 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:07.502949 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:07.502949 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:07.502949 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:07.502949 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:07.502949 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:07.502949 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:07.502949 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:07.502949 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:07.502949 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:07.502949 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:07.502949 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:07.502949 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:07.502949 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:07.502949 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:07.502949 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:07.502949 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:07.503747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:07.503747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:07.503747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:07.503747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:07.503747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:07.503747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:07.503747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:07.503747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:07.503747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:07.503747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:07.503747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:48:07.503747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:07.507826 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 153/306] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:48:07.513547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac33e7c-d9ed-4f0d-871b-754eeaa774cc None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:07.551493 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21879ed3-d405-4a35-a5fb-8f6e42e0354c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:07.566906 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:07.589739 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d1808-f894-495c-a560-0244dfaef17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:07.590879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d1808-f894-495c-a560-0244dfaef17a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:07.591367 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d1808-f894-495c-a560-0244dfaef17a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:07.591833 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d1808-f894-495c-a560-0244dfaef17a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:07.593179 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4d1808-f894-495c-a560-0244dfaef17a None admin] RBAC: Adding query filter params (role.id=43f9a6d53a0b43c6ada3d7d2eea1c7d2, user.id=51d15ee3ffd24d7898ecc729628d7e96, scope.system=all) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:07.593754 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4d1808-f894-495c-a560-0244dfaef17a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:07.595392 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4d1808-f894-495c-a560-0244dfaef17a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:07.602702 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 154/307] 10.222.0.44 () {56 vars in 1308 bytes} [Thu Jun 23 19:48:07 2022] GET /identity/v3/role_assignments?user.id=51d15ee3ffd24d7898ecc729628d7e96&scope.system=all&role.id=43f9a6d53a0b43c6ada3d7d2eea1c7d2 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:07.633306 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac33e7c-d9ed-4f0d-871b-754eeaa774cc None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:07.673859 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713200cd-f80d-4803-bda1-0a59cb7f700e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:07.674961 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713200cd-f80d-4803-bda1-0a59cb7f700e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:07.675437 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713200cd-f80d-4803-bda1-0a59cb7f700e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:07.675921 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713200cd-f80d-4803-bda1-0a59cb7f700e None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:07.676911 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713200cd-f80d-4803-bda1-0a59cb7f700e None admin] RBAC: Adding query filter params (name=member) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:07.677431 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713200cd-f80d-4803-bda1-0a59cb7f700e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:07.679232 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713200cd-f80d-4803-bda1-0a59cb7f700e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:07.685879 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:07.692132 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 154/308] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jun 23 19:48:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:07.699816 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d1808-f894-495c-a560-0244dfaef17a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:07.813221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d1808-f894-495c-a560-0244dfaef17a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:07.849634 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7758d-b6c8-4b2a-910e-3d65d30b2116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:07.850555 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7758d-b6c8-4b2a-910e-3d65d30b2116 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:07.850875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7758d-b6c8-4b2a-910e-3d65d30b2116 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:07.851151 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7758d-b6c8-4b2a-910e-3d65d30b2116 None admin] PATH_INFO: `/v3/users/a9168a7cac5a4622add75e1fb21c4ef7` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:07.852423 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7758d-b6c8-4b2a-910e-3d65d30b2116 None admin] RBAC: Authorizing `identity:get_user(user_id=a9168a7cac5a4622add75e1fb21c4ef7)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:07.853883 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7758d-b6c8-4b2a-910e-3d65d30b2116 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:07.855896 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 155/309] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:48:07 2022] GET /identity/v3/users/a9168a7cac5a4622add75e1fb21c4ef7 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:07.861062 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713200cd-f80d-4803-bda1-0a59cb7f700e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:07.863177 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6214d6d-c1eb-4d38-a050-de0977a63f2e None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:07.863468 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6214d6d-c1eb-4d38-a050-de0977a63f2e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:07.863742 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6214d6d-c1eb-4d38-a050-de0977a63f2e None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:07.865315 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 156/310] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:07.872197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcf087-f3d7-4310-8a93-438d411554b5 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:07.872685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcf087-f3d7-4310-8a93-438d411554b5 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:07.873106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcf087-f3d7-4310-8a93-438d411554b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:07.890222 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-02fcf087-f3d7-4310-8a93-438d411554b5 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:07.891444 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-02fcf087-f3d7-4310-8a93-438d411554b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:07.935068 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:07.971973 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 157/311] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 19:48:07.990434 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fcf087-f3d7-4310-8a93-438d411554b5 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:07.995685 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713200cd-f80d-4803-bda1-0a59cb7f700e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:08.039641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4061ca-886c-42d9-9cbf-a283033369f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:08.041293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4061ca-886c-42d9-9cbf-a283033369f7 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:08.041574 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4061ca-886c-42d9-9cbf-a283033369f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:08.041844 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4061ca-886c-42d9-9cbf-a283033369f7 None admin] PATH_INFO: `/v3/system/users/a9168a7cac5a4622add75e1fb21c4ef7/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:08.043396 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4061ca-886c-42d9-9cbf-a283033369f7 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a9168a7cac5a4622add75e1fb21c4ef7, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:08.045657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4061ca-886c-42d9-9cbf-a283033369f7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:08.103709 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fcf087-f3d7-4310-8a93-438d411554b5 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:08.141461 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1472529b-9eac-4733-90ff-0e6b5e27d50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:08.142255 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472529b-9eac-4733-90ff-0e6b5e27d50f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:08.142534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472529b-9eac-4733-90ff-0e6b5e27d50f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:08.142805 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472529b-9eac-4733-90ff-0e6b5e27d50f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:08.149133 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 155/312] 10.222.0.44 () {58 vars in 1334 bytes} [Thu Jun 23 19:48:07 2022] PUT /identity/v3/system/users/a9168a7cac5a4622add75e1fb21c4ef7/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:48:08.164066 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1472529b-9eac-4733-90ff-0e6b5e27d50f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:08.173276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-1472529b-9eac-4733-90ff-0e6b5e27d50f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 19:48:08.173276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:08.173276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:08.173276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:08.173276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:08.173276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:08.173276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:08.173276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:08.173276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:08.173276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:08.173276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:08.173276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:08.173276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:08.173276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:08.173276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:08.173276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:08.174259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:08.174259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:08.174259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:08.174259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:08.174259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:08.174259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:08.174259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:08.174259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:08.174259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:08.174259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:08.174259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:08.174259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:08.174259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:08.174259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:08.174259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:08.174259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:08.175035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:08.175035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:08.175035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:08.175035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:08.175035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:08.175035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:08.175035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:08.175035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:08.175035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:08.175035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:08.175035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 19:48:08.175035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:08.175035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 158/313] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:48:08.185647 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4061ca-886c-42d9-9cbf-a283033369f7 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:08.270031 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:08.312562 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4061ca-886c-42d9-9cbf-a283033369f7 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:08.347761 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa60b880-943b-4c88-a242-d69e7382eb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:08.348466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60b880-943b-4c88-a242-d69e7382eb64 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:08.348804 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60b880-943b-4c88-a242-d69e7382eb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:08.349082 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60b880-943b-4c88-a242-d69e7382eb64 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:08.349906 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa60b880-943b-4c88-a242-d69e7382eb64 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:08.350870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa60b880-943b-4c88-a242-d69e7382eb64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:08.352246 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa60b880-943b-4c88-a242-d69e7382eb64 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:08.360768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 156/314] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jun 23 19:48:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:08.366651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1472529b-9eac-4733-90ff-0e6b5e27d50f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:08.402422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:08.480957 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1472529b-9eac-4733-90ff-0e6b5e27d50f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:08.518228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c7d84-aede-4dcf-8326-2531cd39931c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:08.519105 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c7d84-aede-4dcf-8326-2531cd39931c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:08.519386 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c7d84-aede-4dcf-8326-2531cd39931c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:08.519663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c7d84-aede-4dcf-8326-2531cd39931c None admin] PATH_INFO: `/v3/users/e67cc18cbc114cd3ba4c38bb9ebb91c8` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:08.520898 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c7d84-aede-4dcf-8326-2531cd39931c None admin] RBAC: Authorizing `identity:get_user(user_id=e67cc18cbc114cd3ba4c38bb9ebb91c8)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:08.522391 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c7d84-aede-4dcf-8326-2531cd39931c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:08.524390 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 159/315] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:48:08 2022] GET /identity/v3/users/e67cc18cbc114cd3ba4c38bb9ebb91c8 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:08.529473 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa60b880-943b-4c88-a242-d69e7382eb64 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:08.637708 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa60b880-943b-4c88-a242-d69e7382eb64 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:08.663348 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:08.672256 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c4cd0-8a60-4199-8462-aa5302811c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:08.672925 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c4cd0-8a60-4199-8462-aa5302811c9e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:08.673254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c4cd0-8a60-4199-8462-aa5302811c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:08.673647 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c4cd0-8a60-4199-8462-aa5302811c9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:08.744851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5c4cd0-8a60-4199-8462-aa5302811c9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:08.746268 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5c4cd0-8a60-4199-8462-aa5302811c9e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:08.853867 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 157/316] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:48:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:48:08.859084 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c7d84-aede-4dcf-8326-2531cd39931c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:08.966460 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c7d84-aede-4dcf-8326-2531cd39931c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:08.968013 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:09.001702 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da983ba5-7d74-4185-b629-4b1d6f3fb27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:09.002483 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da983ba5-7d74-4185-b629-4b1d6f3fb27b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:09.002763 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da983ba5-7d74-4185-b629-4b1d6f3fb27b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:09.003037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da983ba5-7d74-4185-b629-4b1d6f3fb27b None admin] PATH_INFO: `/v3/projects/6c9c90447a664a9fa575e9c13ef585f0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:09.004314 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da983ba5-7d74-4185-b629-4b1d6f3fb27b None admin] RBAC: Authorizing `identity:get_project(project_id=6c9c90447a664a9fa575e9c13ef585f0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:09.005820 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da983ba5-7d74-4185-b629-4b1d6f3fb27b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:09.008447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 160/317] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:48:08 2022] GET /identity/v3/projects/6c9c90447a664a9fa575e9c13ef585f0 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:09.013869 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c4cd0-8a60-4199-8462-aa5302811c9e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:09.043373 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:09.117419 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c4cd0-8a60-4199-8462-aa5302811c9e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:09.151656 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdcb3b0-f67a-4e2a-a123-c4ede593083d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:09.152323 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdcb3b0-f67a-4e2a-a123-c4ede593083d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:09.152608 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdcb3b0-f67a-4e2a-a123-c4ede593083d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:09.152876 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdcb3b0-f67a-4e2a-a123-c4ede593083d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:09.153699 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdcb3b0-f67a-4e2a-a123-c4ede593083d None admin] RBAC: Adding query filter params (role.id=43f9a6d53a0b43c6ada3d7d2eea1c7d2, scope.project.id=6c9c90447a664a9fa575e9c13ef585f0, user.id=e67cc18cbc114cd3ba4c38bb9ebb91c8) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:09.154004 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdcb3b0-f67a-4e2a-a123-c4ede593083d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:09.155458 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdcb3b0-f67a-4e2a-a123-c4ede593083d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:09.161709 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 158/318] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:48:09 2022] GET /identity/v3/role_assignments?user.id=e67cc18cbc114cd3ba4c38bb9ebb91c8&scope.project.id=6c9c90447a664a9fa575e9c13ef585f0&role.id=43f9a6d53a0b43c6ada3d7d2eea1c7d2 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:09.394032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3961ac3-6c96-4f46-969e-52f625b404f1 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:09.394416 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3961ac3-6c96-4f46-969e-52f625b404f1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:09.394578 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3961ac3-6c96-4f46-969e-52f625b404f1 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:09.396228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 161/319] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:09.402996 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fc191-efe8-4b5e-b716-3f3a8f4245e6 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:09.403604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fc191-efe8-4b5e-b716-3f3a8f4245e6 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:09.404136 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fc191-efe8-4b5e-b716-3f3a8f4245e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:09.421134 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-100fc191-efe8-4b5e-b716-3f3a8f4245e6 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:09.422330 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-100fc191-efe8-4b5e-b716-3f3a8f4245e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:09.490478 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 159/320] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:09.513366 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3961ac3-6c96-4f46-969e-52f625b404f1 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:09.619074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3961ac3-6c96-4f46-969e-52f625b404f1 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:09.656678 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30567fba-da94-4906-81ad-924353d30e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:09.657408 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30567fba-da94-4906-81ad-924353d30e32 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:09.657691 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30567fba-da94-4906-81ad-924353d30e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:09.657966 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30567fba-da94-4906-81ad-924353d30e32 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:09.659134 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30567fba-da94-4906-81ad-924353d30e32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:09.660608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30567fba-da94-4906-81ad-924353d30e32 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:09.663414 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 162/321] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:09.669044 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100fc191-efe8-4b5e-b716-3f3a8f4245e6 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:09.690182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:09.723133 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105d4a1-d19a-4836-81f6-ae7944d966a6 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:09.723449 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105d4a1-d19a-4836-81f6-ae7944d966a6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:09.723724 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105d4a1-d19a-4836-81f6-ae7944d966a6 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:09.725172 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 163/322] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:48:09.730209 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc78fe-0244-4592-8c43-f5c5501184c9 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:09.730400 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc78fe-0244-4592-8c43-f5c5501184c9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:09.730680 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc78fe-0244-4592-8c43-f5c5501184c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:09.746791 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c4cc78fe-0244-4592-8c43-f5c5501184c9 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:09.747973 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c4cc78fe-0244-4592-8c43-f5c5501184c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:09.776150 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100fc191-efe8-4b5e-b716-3f3a8f4245e6 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:09.780167 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:09.812596 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19666197-4a5e-44f8-8ecd-4509d362d985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:09.814414 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19666197-4a5e-44f8-8ecd-4509d362d985 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:09.814744 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19666197-4a5e-44f8-8ecd-4509d362d985 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:09.815028 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19666197-4a5e-44f8-8ecd-4509d362d985 None admin] PATH_INFO: `/v3/groups` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:09.815740 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19666197-4a5e-44f8-8ecd-4509d362d985 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:09.817238 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19666197-4a5e-44f8-8ecd-4509d362d985 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:09.821773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 164/323] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 19:48:09.844272 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc78fe-0244-4592-8c43-f5c5501184c9 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:09.908418 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 160/324] 10.222.0.44 () {60 vars in 1134 bytes} [Thu Jun 23 19:48:09 2022] POST /identity/v3/groups => generated 226 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:48:09.963868 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc78fe-0244-4592-8c43-f5c5501184c9 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:10.000145 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4096d-347a-4543-b6ab-82180328f0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:10.000989 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4096d-347a-4543-b6ab-82180328f0cc None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:10.001273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4096d-347a-4543-b6ab-82180328f0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:10.001548 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce4096d-347a-4543-b6ab-82180328f0cc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:10.014063 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce4096d-347a-4543-b6ab-82180328f0cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:10.022080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-4ce4096d-347a-4543-b6ab-82180328f0cc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 19:48:10.022080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:10.022080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:10.022080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:10.022080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:10.022080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:10.022080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:10.022080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:10.022080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:10.022080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:10.022080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:10.022080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:10.022080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:10.022080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:10.022080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:10.022080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:10.023207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:10.023207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:10.023207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:10.023207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:10.023207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:10.023207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:10.023207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:10.023207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:10.023207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:10.023207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:10.023207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:10.023207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:10.023207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:10.023207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:10.023207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:10.023207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:10.024197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:10.024197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:10.024197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:10.024197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:10.024197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:10.024197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:10.024197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:10.024197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:10.024197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:10.024197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:10.024197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 19:48:10.024197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:10.028333 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 165/325] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:48:10.029519 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19666197-4a5e-44f8-8ecd-4509d362d985 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:10.115381 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:10.135121 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19666197-4a5e-44f8-8ecd-4509d362d985 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:10.171258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f2f4ff-127c-44a5-899d-9f1262bbdf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:10.172980 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2f4ff-127c-44a5-899d-9f1262bbdf1b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:10.173739 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2f4ff-127c-44a5-899d-9f1262bbdf1b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:10.174823 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f2f4ff-127c-44a5-899d-9f1262bbdf1b None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:10.176247 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2f4ff-127c-44a5-899d-9f1262bbdf1b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:10.176821 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2f4ff-127c-44a5-899d-9f1262bbdf1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:10.179853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f2f4ff-127c-44a5-899d-9f1262bbdf1b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:10.194608 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 161/326] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jun 23 19:48:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:10.200104 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4096d-347a-4543-b6ab-82180328f0cc None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:10.231446 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:10.310351 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce4096d-347a-4543-b6ab-82180328f0cc None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:10.347860 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78848405-feb8-4fb6-8cac-b963b274d8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:10.348842 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78848405-feb8-4fb6-8cac-b963b274d8db None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:10.349204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78848405-feb8-4fb6-8cac-b963b274d8db None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:10.349481 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78848405-feb8-4fb6-8cac-b963b274d8db None admin] PATH_INFO: `/v3/users/51d15ee3ffd24d7898ecc729628d7e96` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:10.350809 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2676f-87ae-4844-8766-a3f53227f676 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:10.350933 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78848405-feb8-4fb6-8cac-b963b274d8db None admin] RBAC: Authorizing `identity:get_user(user_id=51d15ee3ffd24d7898ecc729628d7e96)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:10.351099 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2676f-87ae-4844-8766-a3f53227f676 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:10.351379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2676f-87ae-4844-8766-a3f53227f676 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:10.352530 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78848405-feb8-4fb6-8cac-b963b274d8db None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:10.353180 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 162/327] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:10.354256 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 166/328] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:48:10 2022] GET /identity/v3/users/51d15ee3ffd24d7898ecc729628d7e96 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:10.358322 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ce25e-c0cd-4937-82d1-fdc926f2113c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:10.358688 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ce25e-c0cd-4937-82d1-fdc926f2113c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:10.358964 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ce25e-c0cd-4937-82d1-fdc926f2113c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:10.359475 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78848405-feb8-4fb6-8cac-b963b274d8db None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:10.385890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-830ce25e-c0cd-4937-82d1-fdc926f2113c None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:10.387568 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-830ce25e-c0cd-4937-82d1-fdc926f2113c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:10.493747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 163/329] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 19:48:10.517650 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ce25e-c0cd-4937-82d1-fdc926f2113c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:10.517869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:10.525195 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78848405-feb8-4fb6-8cac-b963b274d8db None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:10.567694 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9501d5d-7d19-4a21-bad2-1c24c91a5151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:10.568595 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9501d5d-7d19-4a21-bad2-1c24c91a5151 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:10.568930 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9501d5d-7d19-4a21-bad2-1c24c91a5151 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:10.569207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9501d5d-7d19-4a21-bad2-1c24c91a5151 None admin] PATH_INFO: `/v3/system/users/51d15ee3ffd24d7898ecc729628d7e96/roles/43f9a6d53a0b43c6ada3d7d2eea1c7d2` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:10.571383 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9501d5d-7d19-4a21-bad2-1c24c91a5151 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=51d15ee3ffd24d7898ecc729628d7e96, role_id=43f9a6d53a0b43c6ada3d7d2eea1c7d2)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:10.572816 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9501d5d-7d19-4a21-bad2-1c24c91a5151 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:10.626503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:10.649845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 167/330] 10.222.0.44 () {58 vars in 1334 bytes} [Thu Jun 23 19:48:10 2022] PUT /identity/v3/system/users/51d15ee3ffd24d7898ecc729628d7e96/roles/43f9a6d53a0b43c6ada3d7d2eea1c7d2 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:48:10.667667 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ce25e-c0cd-4937-82d1-fdc926f2113c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:10.720155 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac132067-088c-4a81-9fa8-38ed4d0d97e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:10.720993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac132067-088c-4a81-9fa8-38ed4d0d97e9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:10.721274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac132067-088c-4a81-9fa8-38ed4d0d97e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:10.721546 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac132067-088c-4a81-9fa8-38ed4d0d97e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:10.737052 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac132067-088c-4a81-9fa8-38ed4d0d97e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:10.745110 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-ac132067-088c-4a81-9fa8-38ed4d0d97e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:48:10.745110 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:10.745110 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:10.745110 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:10.745110 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:10.745110 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:10.745110 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:10.745110 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:10.745110 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:10.745110 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:10.745110 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:10.745110 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:10.745110 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:10.745110 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:10.745110 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:10.745110 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:10.746030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:10.746030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:10.746030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:10.746030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:10.746030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:10.746030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:10.746030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:10.746030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:10.746030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:10.746030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:10.746030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:10.746030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:10.746030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:10.746030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:10.746030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:10.746030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:10.747455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:10.747455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:10.747455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:10.747455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:10.747455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:10.747455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:10.747455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:10.747455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:10.747455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:10.747455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:10.747455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:48:10.747455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:10.747455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 164/331] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:48:10.752248 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9501d5d-7d19-4a21-bad2-1c24c91a5151 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:10.901334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9501d5d-7d19-4a21-bad2-1c24c91a5151 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:10.940282 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb797f-b17b-4617-a063-b7fe71f1e815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:10.940981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb797f-b17b-4617-a063-b7fe71f1e815 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:10.941261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb797f-b17b-4617-a063-b7fe71f1e815 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:10.941532 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb797f-b17b-4617-a063-b7fe71f1e815 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:10.942337 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb797f-b17b-4617-a063-b7fe71f1e815 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:10.942644 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb797f-b17b-4617-a063-b7fe71f1e815 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:10.944620 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb797f-b17b-4617-a063-b7fe71f1e815 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:10.953542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 168/332] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jun 23 19:48:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:10.959086 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac132067-088c-4a81-9fa8-38ed4d0d97e9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:11.085970 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac132067-088c-4a81-9fa8-38ed4d0d97e9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:11.126342 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:11.133033 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65520e9d-0941-4a79-9b55-dea30287d21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:11.133940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65520e9d-0941-4a79-9b55-dea30287d21a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:11.134537 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65520e9d-0941-4a79-9b55-dea30287d21a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:11.134812 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65520e9d-0941-4a79-9b55-dea30287d21a None admin] PATH_INFO: `/v3/users/a9168a7cac5a4622add75e1fb21c4ef7` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:11.136098 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65520e9d-0941-4a79-9b55-dea30287d21a None admin] RBAC: Authorizing `identity:get_user(user_id=a9168a7cac5a4622add75e1fb21c4ef7)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:11.141954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65520e9d-0941-4a79-9b55-dea30287d21a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:11.143363 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 165/333] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:48:10 2022] GET /identity/v3/users/a9168a7cac5a4622add75e1fb21c4ef7 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:11.150390 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb797f-b17b-4617-a063-b7fe71f1e815 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:11.194322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:11.293603 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb797f-b17b-4617-a063-b7fe71f1e815 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:11.347560 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1daa23-867a-49f2-8b9b-dbb706dac721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:11.348133 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1daa23-867a-49f2-8b9b-dbb706dac721 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:11.348439 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1daa23-867a-49f2-8b9b-dbb706dac721 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:11.348715 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1daa23-867a-49f2-8b9b-dbb706dac721 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:11.349550 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1daa23-867a-49f2-8b9b-dbb706dac721 None admin] RBAC: Adding query filter params (role.id=860491bfbb574772a7a213d6a25962e0, user.id=a9168a7cac5a4622add75e1fb21c4ef7, scope.system=all) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:11.349862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1daa23-867a-49f2-8b9b-dbb706dac721 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:11.351955 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1daa23-867a-49f2-8b9b-dbb706dac721 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:11.367362 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 169/334] 10.222.0.44 () {56 vars in 1308 bytes} [Thu Jun 23 19:48:11 2022] GET /identity/v3/role_assignments?user.id=a9168a7cac5a4622add75e1fb21c4ef7&scope.system=all&role.id=860491bfbb574772a7a213d6a25962e0 => generated 510 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:12.886278 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e3a23-3ec9-496b-ad2a-5c1fd28c523f None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:12.887403 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e3a23-3ec9-496b-ad2a-5c1fd28c523f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:12.887992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e3a23-3ec9-496b-ad2a-5c1fd28c523f None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:12.889670 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 166/335] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:12.895676 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638661f-5486-492b-8762-de7c8899ff4d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:12.896549 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638661f-5486-492b-8762-de7c8899ff4d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:12.897077 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4638661f-5486-492b-8762-de7c8899ff4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:12.925306 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4638661f-5486-492b-8762-de7c8899ff4d None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:12.926842 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4638661f-5486-492b-8762-de7c8899ff4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:13.002127 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 170/336] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 19:48:13.033613 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e3a23-3ec9-496b-ad2a-5c1fd28c523f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:13.133894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2921c-e48d-4a47-bab5-ddbedea42f98 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:13.134610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2921c-e48d-4a47-bab5-ddbedea42f98 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:13.135094 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2921c-e48d-4a47-bab5-ddbedea42f98 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:13.136761 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 171/337] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:48:13.143710 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341a5bb-827e-4660-86b1-92231f3b4169 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:13.143997 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341a5bb-827e-4660-86b1-92231f3b4169 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:13.144274 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2341a5bb-827e-4660-86b1-92231f3b4169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:13.161609 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e3a23-3ec9-496b-ad2a-5c1fd28c523f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:13.168846 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-2341a5bb-827e-4660-86b1-92231f3b4169 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:13.174511 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2341a5bb-827e-4660-86b1-92231f3b4169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:13.180226 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:13.238090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92065720-b586-4ff7-b365-0a28b059a35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:13.239459 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92065720-b586-4ff7-b365-0a28b059a35d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:13.240448 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92065720-b586-4ff7-b365-0a28b059a35d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:13.241137 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92065720-b586-4ff7-b365-0a28b059a35d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:13.267113 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92065720-b586-4ff7-b365-0a28b059a35d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:13.279302 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-92065720-b586-4ff7-b365-0a28b059a35d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:48:13.279302 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:13.279302 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:13.279302 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:13.279302 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:13.279302 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:13.279302 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:13.279302 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:13.279302 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:13.279302 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:13.279302 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:13.279302 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:13.279302 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:13.279302 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:13.279302 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:13.279302 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:13.280182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:13.280182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:13.280182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:13.280182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:13.280182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:13.280182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:13.280182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:13.280182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:13.280182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:13.280182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:13.280182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:13.280182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:13.280182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:13.280182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:13.280182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:13.280182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:13.280993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:13.280993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:13.280993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:13.280993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:13.280993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:13.280993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:13.280993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:13.280993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:13.280993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:13.280993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:13.280993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:48:13.280993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:13.291191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 167/338] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:13.299434 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92065720-b586-4ff7-b365-0a28b059a35d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:13.346045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 172/339] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:13.405356 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:13.405649 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341a5bb-827e-4660-86b1-92231f3b4169 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:13.466041 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92065720-b586-4ff7-b365-0a28b059a35d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:13.510978 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e54d08-8c4c-4627-8892-19dc5aa8276e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:13.512263 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e54d08-8c4c-4627-8892-19dc5aa8276e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:13.512776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e54d08-8c4c-4627-8892-19dc5aa8276e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:13.513659 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e54d08-8c4c-4627-8892-19dc5aa8276e None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:13.514753 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e54d08-8c4c-4627-8892-19dc5aa8276e None admin] RBAC: Adding query filter params (name=member) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:13.515277 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e54d08-8c4c-4627-8892-19dc5aa8276e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:13.517103 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e54d08-8c4c-4627-8892-19dc5aa8276e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:13.528387 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 168/340] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jun 23 19:48:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:13.538898 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e54d08-8c4c-4627-8892-19dc5aa8276e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:13.577251 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2341a5bb-827e-4660-86b1-92231f3b4169 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:13.611852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a0392-a210-4eb1-823c-96f19ed5c223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:13.614544 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a0392-a210-4eb1-823c-96f19ed5c223 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:13.614830 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a0392-a210-4eb1-823c-96f19ed5c223 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:13.615108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a0392-a210-4eb1-823c-96f19ed5c223 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:13.623061 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:13.630920 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a0392-a210-4eb1-823c-96f19ed5c223 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:13.639447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-d47a0392-a210-4eb1-823c-96f19ed5c223 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:48:13.639447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:13.639447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:13.639447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:13.639447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:13.639447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:13.639447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:13.639447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:13.639447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:13.639447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:13.639447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:13.639447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:13.639447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:13.639447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:13.639447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:13.639447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:13.640369 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:13.640369 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:13.640369 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:13.640369 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:13.640369 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:13.640369 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:13.640369 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:13.640369 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:13.640369 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:13.640369 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:13.640369 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:13.640369 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:13.640369 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:13.640369 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:13.640369 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:13.640369 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:13.641181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:13.641181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:13.641181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:13.641181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:13.641181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:13.641181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:13.641181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:13.641181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:13.641181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:13.641181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:13.641181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:48:13.641181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:13.643150 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 173/341] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:48:13.649522 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a0392-a210-4eb1-823c-96f19ed5c223 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:13.704998 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e54d08-8c4c-4627-8892-19dc5aa8276e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:13.757527 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:13.778930 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb829c79-9aeb-4ca4-900f-8cef04c4bd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:13.779582 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb829c79-9aeb-4ca4-900f-8cef04c4bd48 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:13.779860 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb829c79-9aeb-4ca4-900f-8cef04c4bd48 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:13.780131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb829c79-9aeb-4ca4-900f-8cef04c4bd48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:13.788968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a0392-a210-4eb1-823c-96f19ed5c223 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:13.839757 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3846a8f-bf5b-4948-ba44-f35502acfad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:13.840452 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3846a8f-bf5b-4948-ba44-f35502acfad1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:13.840734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3846a8f-bf5b-4948-ba44-f35502acfad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:13.841001 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3846a8f-bf5b-4948-ba44-f35502acfad1 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:13.841749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3846a8f-bf5b-4948-ba44-f35502acfad1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:13.842051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3846a8f-bf5b-4948-ba44-f35502acfad1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:13.843430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3846a8f-bf5b-4948-ba44-f35502acfad1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:13.855420 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 174/342] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jun 23 19:48:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:13.861518 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23831547-4f58-4242-85e6-2bd9d521dfc3 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:13.861802 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23831547-4f58-4242-85e6-2bd9d521dfc3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:13.862072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23831547-4f58-4242-85e6-2bd9d521dfc3 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:13.863491 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 175/343] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:13.864454 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1c233-8f6e-44f8-8e2a-a3b14d08417a None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:13.864794 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1c233-8f6e-44f8-8e2a-a3b14d08417a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:13.865067 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1c233-8f6e-44f8-8e2a-a3b14d08417a None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:13.866809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 176/344] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:13.867492 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a1c233-8f6e-44f8-8e2a-a3b14d08417a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:13.885888 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb829c79-9aeb-4ca4-900f-8cef04c4bd48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:13.887746 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb829c79-9aeb-4ca4-900f-8cef04c4bd48 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:13.934569 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:13.991222 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a1c233-8f6e-44f8-8e2a-a3b14d08417a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:14.023899 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 169/345] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:48:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:48:14.025646 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10aba1e-95b0-4503-a949-1ae4510b3364 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:14.026112 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10aba1e-95b0-4503-a949-1ae4510b3364 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:14.026580 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10aba1e-95b0-4503-a949-1ae4510b3364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:14.037535 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b75e1-b89c-4ac5-a643-de8c161e5cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:14.038484 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b75e1-b89c-4ac5-a643-de8c161e5cdb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:14.038986 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b75e1-b89c-4ac5-a643-de8c161e5cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:14.039973 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b75e1-b89c-4ac5-a643-de8c161e5cdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:14.046260 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f10aba1e-95b0-4503-a949-1ae4510b3364 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:14.047449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f10aba1e-95b0-4503-a949-1ae4510b3364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:14.068230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:14.132094 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 170/346] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:14.133527 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d066b2-4385-4b04-8750-cbed1394d930 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:14.133772 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d066b2-4385-4b04-8750-cbed1394d930 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:14.134045 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d066b2-4385-4b04-8750-cbed1394d930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:14.136129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b75e1-b89c-4ac5-a643-de8c161e5cdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:14.137830 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b75e1-b89c-4ac5-a643-de8c161e5cdb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:14.151374 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-23d066b2-4385-4b04-8750-cbed1394d930 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:14.153124 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-23d066b2-4385-4b04-8750-cbed1394d930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:14.222836 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:14.233746 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 171/347] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:14.235299 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d066b2-4385-4b04-8750-cbed1394d930 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:14.286268 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 177/348] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:48:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:48:14.287091 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b75e1-b89c-4ac5-a643-de8c161e5cdb None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:14.328317 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:14.347465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d066b2-4385-4b04-8750-cbed1394d930 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:14.388389 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e559d6c-bade-4e40-985a-d5bcb9a5505a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:14.389693 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e559d6c-bade-4e40-985a-d5bcb9a5505a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:14.390151 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e559d6c-bade-4e40-985a-d5bcb9a5505a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:14.390594 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e559d6c-bade-4e40-985a-d5bcb9a5505a None admin] PATH_INFO: `/v3/projects/4c94dfce039f4ff29bec2f9831ec1748` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:14.391999 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e559d6c-bade-4e40-985a-d5bcb9a5505a None admin] RBAC: Authorizing `identity:get_project(project_id=4c94dfce039f4ff29bec2f9831ec1748)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:14.393951 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e559d6c-bade-4e40-985a-d5bcb9a5505a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:14.395874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 172/349] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:48:14 2022] GET /identity/v3/projects/4c94dfce039f4ff29bec2f9831ec1748 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:14.400421 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e559d6c-bade-4e40-985a-d5bcb9a5505a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:14.407696 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b75e1-b89c-4ac5-a643-de8c161e5cdb None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:14.453268 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6bfff-06c2-4956-892f-9e41679cfbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:14.454248 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6bfff-06c2-4956-892f-9e41679cfbe0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:14.454632 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6bfff-06c2-4956-892f-9e41679cfbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:14.454912 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6bfff-06c2-4956-892f-9e41679cfbe0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:14.469362 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc6bfff-06c2-4956-892f-9e41679cfbe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:14.479996 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-ddc6bfff-06c2-4956-892f-9e41679cfbe0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 19:48:14.479996 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:14.479996 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:14.479996 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:14.479996 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:14.479996 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:14.479996 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:14.479996 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:14.479996 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:14.479996 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:14.479996 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:14.479996 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:14.479996 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:14.479996 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:14.479996 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:14.479996 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:14.481045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:14.481045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:14.481045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:14.481045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:14.481045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:14.481045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:14.481045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:14.481045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:14.481045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:14.481045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:14.481045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:14.481045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:14.481045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:14.481045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:14.481045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:14.481045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:14.482041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:14.482041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:14.482041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:14.482041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:14.482041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:14.482041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:14.482041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:14.482041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:14.482041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:14.482041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:14.482041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 19:48:14.482041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:14.482041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 178/350] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jun 23 19:48:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:14.482795 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6bfff-06c2-4956-892f-9e41679cfbe0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:14.513232 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:14.532281 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e559d6c-bade-4e40-985a-d5bcb9a5505a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:14.585002 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48f8f0-1211-4891-bc60-852d3e77f2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:14.585002 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48f8f0-1211-4891-bc60-852d3e77f2d2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:14.585002 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48f8f0-1211-4891-bc60-852d3e77f2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:14.585002 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48f8f0-1211-4891-bc60-852d3e77f2d2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:14.596930 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f48f8f0-1211-4891-bc60-852d3e77f2d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:14.604532 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6bfff-06c2-4956-892f-9e41679cfbe0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:14.606261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-1f48f8f0-1211-4891-bc60-852d3e77f2d2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 19:48:14.606261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:14.606261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:14.606261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:14.606261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:14.606261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:14.606261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:14.606261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:14.606261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:14.606261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:14.606261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:14.606261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:14.606261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:14.606261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:14.606261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:14.606261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:14.607298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:14.607298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:14.607298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:14.607298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:14.607298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:14.607298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:14.607298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:14.607298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:14.607298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:14.607298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:14.607298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:14.607298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:14.607298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:14.607298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:14.607298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:14.607298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:14.608267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:14.608267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:14.608267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:14.608267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:14.608267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:14.608267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:14.608267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:14.608267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:14.608267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:14.608267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:14.608267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 19:48:14.608267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:14.610338 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 173/351] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jun 23 19:48:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:14.611272 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48f8f0-1211-4891-bc60-852d3e77f2d2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:14.625501 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:14.649168 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c701f-ad45-40de-a237-5583d28fa20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:14.650041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c701f-ad45-40de-a237-5583d28fa20c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:14.650474 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c701f-ad45-40de-a237-5583d28fa20c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:14.650757 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c701f-ad45-40de-a237-5583d28fa20c None admin] PATH_INFO: `/v3/projects/6c9c90447a664a9fa575e9c13ef585f0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:14.651975 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c701f-ad45-40de-a237-5583d28fa20c None admin] RBAC: Authorizing `identity:get_project(project_id=6c9c90447a664a9fa575e9c13ef585f0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:14.653478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c701f-ad45-40de-a237-5583d28fa20c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:14.657654 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 179/352] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:48:14 2022] GET /identity/v3/projects/6c9c90447a664a9fa575e9c13ef585f0 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:14.657654 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c701f-ad45-40de-a237-5583d28fa20c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:14.742948 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48f8f0-1211-4891-bc60-852d3e77f2d2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:14.762942 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:14.773931 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c701f-ad45-40de-a237-5583d28fa20c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:14.784890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1a49c-eb32-4f21-a39f-3d4486ec3e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:14.786443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1a49c-eb32-4f21-a39f-3d4486ec3e76 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:14.786551 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1a49c-eb32-4f21-a39f-3d4486ec3e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:14.786806 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1a49c-eb32-4f21-a39f-3d4486ec3e76 None admin] PATH_INFO: `/v3/groups/f5e4c696c29b4f179b9957e5743c09b1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:14.793196 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a1a49c-eb32-4f21-a39f-3d4486ec3e76 None admin] RBAC: Authorizing `identity:get_group(group_id=f5e4c696c29b4f179b9957e5743c09b1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:14.794681 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a1a49c-eb32-4f21-a39f-3d4486ec3e76 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:14.797016 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 174/353] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jun 23 19:48:14 2022] GET /identity/v3/groups/f5e4c696c29b4f179b9957e5743c09b1 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:14.798265 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21aeb1-7dd8-495d-b60a-e2ca2471f3c3 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:14.798552 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21aeb1-7dd8-495d-b60a-e2ca2471f3c3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:14.798831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21aeb1-7dd8-495d-b60a-e2ca2471f3c3 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:14.800333 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 175/354] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:14.800973 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21aeb1-7dd8-495d-b60a-e2ca2471f3c3 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:14.816372 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35d51f-94cf-4d5b-8aa7-27d2ef33ea1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:14.817045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35d51f-94cf-4d5b-8aa7-27d2ef33ea1d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:14.817326 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35d51f-94cf-4d5b-8aa7-27d2ef33ea1d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:14.817598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35d51f-94cf-4d5b-8aa7-27d2ef33ea1d None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:14.818341 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35d51f-94cf-4d5b-8aa7-27d2ef33ea1d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:14.818696 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35d51f-94cf-4d5b-8aa7-27d2ef33ea1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:14.820631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35d51f-94cf-4d5b-8aa7-27d2ef33ea1d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:14.832375 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 180/355] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:48:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:14.833947 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057e69c-f79e-4114-ab36-cf6a97501542 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:14.834335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057e69c-f79e-4114-ab36-cf6a97501542 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:14.834677 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057e69c-f79e-4114-ab36-cf6a97501542 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:14.836041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 181/356] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:14.836911 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057e69c-f79e-4114-ab36-cf6a97501542 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:14.890383 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:14.918943 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21aeb1-7dd8-495d-b60a-e2ca2471f3c3 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:14.959559 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b462d87-73b4-462b-acc4-572838058f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:14.960281 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b462d87-73b4-462b-acc4-572838058f57 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:14.960474 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057e69c-f79e-4114-ab36-cf6a97501542 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:14.960896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b462d87-73b4-462b-acc4-572838058f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:14.960896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b462d87-73b4-462b-acc4-572838058f57 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:14.961660 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b462d87-73b4-462b-acc4-572838058f57 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:14.962159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b462d87-73b4-462b-acc4-572838058f57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:14.964508 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b462d87-73b4-462b-acc4-572838058f57 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:14.974491 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 176/357] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:48:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:14.975783 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b462d87-73b4-462b-acc4-572838058f57 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:14.999498 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586eab3-d5d7-4cd4-9ac2-f47a96ba885f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:15.000403 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586eab3-d5d7-4cd4-9ac2-f47a96ba885f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:15.000908 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586eab3-d5d7-4cd4-9ac2-f47a96ba885f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:15.001927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586eab3-d5d7-4cd4-9ac2-f47a96ba885f None admin] PATH_INFO: `/v3/groups/f5e4c696c29b4f179b9957e5743c09b1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:15.002895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586eab3-d5d7-4cd4-9ac2-f47a96ba885f None admin] RBAC: Authorizing `identity:get_group(group_id=f5e4c696c29b4f179b9957e5743c09b1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:15.004374 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586eab3-d5d7-4cd4-9ac2-f47a96ba885f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:15.006032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 182/358] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jun 23 19:48:14 2022] GET /identity/v3/groups/f5e4c696c29b4f179b9957e5743c09b1 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:15.007221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed2aa2-a606-427b-87f7-8514e8014712 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:15.007504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed2aa2-a606-427b-87f7-8514e8014712 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:15.007796 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ed2aa2-a606-427b-87f7-8514e8014712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:15.027338 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b3ed2aa2-a606-427b-87f7-8514e8014712 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:15.030517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b3ed2aa2-a606-427b-87f7-8514e8014712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:15.044477 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:15.095277 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b462d87-73b4-462b-acc4-572838058f57 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:15.103053 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 183/359] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:15.103986 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed2aa2-a606-427b-87f7-8514e8014712 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:15.137379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7197ab-48c1-488a-85d9-ebfda99141e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:15.138040 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7197ab-48c1-488a-85d9-ebfda99141e1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:15.138326 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7197ab-48c1-488a-85d9-ebfda99141e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:15.138611 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7197ab-48c1-488a-85d9-ebfda99141e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:15.139443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7197ab-48c1-488a-85d9-ebfda99141e1 None admin] RBAC: Adding query filter params (group.id=f5e4c696c29b4f179b9957e5743c09b1, role.id=860491bfbb574772a7a213d6a25962e0, scope.project.id=4c94dfce039f4ff29bec2f9831ec1748) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:15.139763 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7197ab-48c1-488a-85d9-ebfda99141e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:15.143967 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7197ab-48c1-488a-85d9-ebfda99141e1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:15.145460 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:15.149159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 177/360] 10.222.0.44 () {56 vars in 1376 bytes} [Thu Jun 23 19:48:14 2022] GET /identity/v3/role_assignments?group.id=f5e4c696c29b4f179b9957e5743c09b1&scope.project.id=4c94dfce039f4ff29bec2f9831ec1748&role.id=860491bfbb574772a7a213d6a25962e0 => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:15.150212 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6cad5b-5237-4401-af0c-bdba997dc58d None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:15.150502 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6cad5b-5237-4401-af0c-bdba997dc58d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:15.150786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6cad5b-5237-4401-af0c-bdba997dc58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:15.174394 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-5f6cad5b-5237-4401-af0c-bdba997dc58d None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:15.175224 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5f6cad5b-5237-4401-af0c-bdba997dc58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:15.250217 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ed2aa2-a606-427b-87f7-8514e8014712 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:15.272364 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 178/361] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:15.272364 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6cad5b-5237-4401-af0c-bdba997dc58d None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:15.293231 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90484c-9dfb-40ba-b392-11002cf9db8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:15.293905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90484c-9dfb-40ba-b392-11002cf9db8b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:15.294193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90484c-9dfb-40ba-b392-11002cf9db8b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:15.294474 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90484c-9dfb-40ba-b392-11002cf9db8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:15.319342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:15.389634 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe90484c-9dfb-40ba-b392-11002cf9db8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:15.391217 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe90484c-9dfb-40ba-b392-11002cf9db8b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:15.400163 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6cad5b-5237-4401-af0c-bdba997dc58d None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:15.438952 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:15.447653 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797f5a0-e39c-48b3-a387-21ba052122d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:15.448799 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797f5a0-e39c-48b3-a387-21ba052122d2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:15.449268 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797f5a0-e39c-48b3-a387-21ba052122d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:15.450142 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9797f5a0-e39c-48b3-a387-21ba052122d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:15.516897 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 184/362] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:48:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:48:15.517771 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90484c-9dfb-40ba-b392-11002cf9db8b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:15.538750 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797f5a0-e39c-48b3-a387-21ba052122d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:15.540262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9797f5a0-e39c-48b3-a387-21ba052122d2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:15.589255 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:15.654221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90484c-9dfb-40ba-b392-11002cf9db8b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:15.670201 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 179/363] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:48:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:48:15.670949 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797f5a0-e39c-48b3-a387-21ba052122d2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:15.705590 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3678c-1038-42b8-86ec-ef1e06621868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:15.706263 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3678c-1038-42b8-86ec-ef1e06621868 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:15.706555 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3678c-1038-42b8-86ec-ef1e06621868 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:15.706824 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3678c-1038-42b8-86ec-ef1e06621868 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:15.707667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3678c-1038-42b8-86ec-ef1e06621868 None admin] RBAC: Adding query filter params (group.id=f5e4c696c29b4f179b9957e5743c09b1, role.id=860491bfbb574772a7a213d6a25962e0, scope.project.id=6c9c90447a664a9fa575e9c13ef585f0) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:15.707976 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3678c-1038-42b8-86ec-ef1e06621868 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:15.711826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3678c-1038-42b8-86ec-ef1e06621868 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:15.715698 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:15.724333 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 185/364] 10.222.0.44 () {56 vars in 1376 bytes} [Thu Jun 23 19:48:15 2022] GET /identity/v3/role_assignments?group.id=f5e4c696c29b4f179b9957e5743c09b1&scope.project.id=6c9c90447a664a9fa575e9c13ef585f0&role.id=860491bfbb574772a7a213d6a25962e0 => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:15.724333 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3678c-1038-42b8-86ec-ef1e06621868 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:15.840820 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9797f5a0-e39c-48b3-a387-21ba052122d2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:15.885543 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3678c-1038-42b8-86ec-ef1e06621868 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:15.892454 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335b0f8-a59a-4d32-ab41-86d25dc1cfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:15.893272 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335b0f8-a59a-4d32-ab41-86d25dc1cfba None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:15.893565 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335b0f8-a59a-4d32-ab41-86d25dc1cfba None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:15.893843 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335b0f8-a59a-4d32-ab41-86d25dc1cfba None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:15.905315 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:15.907636 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3335b0f8-a59a-4d32-ab41-86d25dc1cfba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:15.916441 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-3335b0f8-a59a-4d32-ab41-86d25dc1cfba None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 19:48:15.916441 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:15.916441 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:15.916441 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:15.916441 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:15.916441 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:15.916441 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:15.916441 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:15.916441 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:15.916441 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:15.916441 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:15.916441 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:15.916441 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:15.916441 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:15.916441 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:15.916441 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:15.933333 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:15.933333 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:15.933333 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:15.933333 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:15.933333 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:15.933333 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:15.933333 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:15.933333 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:15.933333 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:15.933333 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:15.933333 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:15.933333 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:15.933333 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:15.933333 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:15.933333 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:15.933333 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:15.934125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:15.934125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:15.934125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:15.934125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:15.934125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:15.934125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:15.934125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:15.934125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:15.934125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:15.934125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:15.934125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 19:48:15.934125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:15.934125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 180/365] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:15.934125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335b0f8-a59a-4d32-ab41-86d25dc1cfba None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:15.935993 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c97878-765b-4636-845a-b9d34415b84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:15.936972 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c97878-765b-4636-845a-b9d34415b84c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:15.937376 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c97878-765b-4636-845a-b9d34415b84c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:15.938206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c97878-765b-4636-845a-b9d34415b84c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:15.950603 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c97878-765b-4636-845a-b9d34415b84c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:15.959325 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-71c97878-765b-4636-845a-b9d34415b84c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:48:15.959325 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:15.959325 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:15.959325 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:15.959325 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:15.959325 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:15.959325 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:15.959325 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:15.959325 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:15.959325 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:15.959325 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:15.959325 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:15.959325 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:15.959325 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:15.959325 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:15.959325 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:15.960472 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:15.960472 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:15.960472 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:15.960472 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:15.960472 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:15.960472 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:15.960472 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:15.960472 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:15.960472 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:15.960472 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:15.960472 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:15.960472 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:15.960472 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:15.960472 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:15.960472 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:15.960472 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:15.961890 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:15.961890 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:15.961890 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:15.961890 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:15.961890 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:15.961890 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:15.961890 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:15.961890 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:15.961890 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:15.961890 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:15.961890 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:48:15.961890 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:15.963998 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 186/366] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:48:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:15.965081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c97878-765b-4636-845a-b9d34415b84c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:16.043931 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:16.057217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335b0f8-a59a-4d32-ab41-86d25dc1cfba None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:16.088885 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c97878-765b-4636-845a-b9d34415b84c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:16.096971 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a5a50-2979-435e-b9b3-339223f40648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:16.097970 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a5a50-2979-435e-b9b3-339223f40648 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:16.098344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a5a50-2979-435e-b9b3-339223f40648 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:16.098732 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a5a50-2979-435e-b9b3-339223f40648 None admin] PATH_INFO: `/v3/projects/4c94dfce039f4ff29bec2f9831ec1748` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:16.100873 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a5a50-2979-435e-b9b3-339223f40648 None admin] RBAC: Authorizing `identity:get_project(project_id=4c94dfce039f4ff29bec2f9831ec1748)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:16.102422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a5a50-2979-435e-b9b3-339223f40648 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:16.104871 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 181/367] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:48:15 2022] GET /identity/v3/projects/4c94dfce039f4ff29bec2f9831ec1748 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:16.105726 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a5a50-2979-435e-b9b3-339223f40648 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:16.135069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62278bde-d508-4d6a-8f6e-6e4c2329569c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:16.135995 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62278bde-d508-4d6a-8f6e-6e4c2329569c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:16.136431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62278bde-d508-4d6a-8f6e-6e4c2329569c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:16.136815 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62278bde-d508-4d6a-8f6e-6e4c2329569c None admin] PATH_INFO: `/v3/projects/6c9c90447a664a9fa575e9c13ef585f0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:16.138178 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62278bde-d508-4d6a-8f6e-6e4c2329569c None admin] RBAC: Authorizing `identity:get_project(project_id=6c9c90447a664a9fa575e9c13ef585f0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:16.139714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62278bde-d508-4d6a-8f6e-6e4c2329569c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:16.141576 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 187/368] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:48:15 2022] GET /identity/v3/projects/6c9c90447a664a9fa575e9c13ef585f0 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:16.142413 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62278bde-d508-4d6a-8f6e-6e4c2329569c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:16.192482 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:16.228649 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a5a50-2979-435e-b9b3-339223f40648 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:16.257629 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62278bde-d508-4d6a-8f6e-6e4c2329569c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:16.265674 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3b683-e957-4521-a34a-2f787207ad7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:16.266413 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3b683-e957-4521-a34a-2f787207ad7f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:16.266707 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3b683-e957-4521-a34a-2f787207ad7f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:16.266986 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3b683-e957-4521-a34a-2f787207ad7f None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:16.267745 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f3b683-e957-4521-a34a-2f787207ad7f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:16.268057 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f3b683-e957-4521-a34a-2f787207ad7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:16.269525 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f3b683-e957-4521-a34a-2f787207ad7f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:16.279917 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 182/369] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jun 23 19:48:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:16.280575 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3b683-e957-4521-a34a-2f787207ad7f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:16.297883 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e82b89-f8f0-4d63-b94f-8339d4936ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:16.298621 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e82b89-f8f0-4d63-b94f-8339d4936ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:16.298952 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e82b89-f8f0-4d63-b94f-8339d4936ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:16.299310 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e82b89-f8f0-4d63-b94f-8339d4936ba0 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:16.300063 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e82b89-f8f0-4d63-b94f-8339d4936ba0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:16.300397 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e82b89-f8f0-4d63-b94f-8339d4936ba0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:16.301817 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e82b89-f8f0-4d63-b94f-8339d4936ba0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:16.307301 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:16.310570 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 188/370] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:16.311436 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e82b89-f8f0-4d63-b94f-8339d4936ba0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:16.417191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3b683-e957-4521-a34a-2f787207ad7f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:16.487643 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e82b89-f8f0-4d63-b94f-8339d4936ba0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:16.490025 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ab910-66dc-4a6e-b0de-9ed729468381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:16.491729 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ab910-66dc-4a6e-b0de-9ed729468381 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:16.492019 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ab910-66dc-4a6e-b0de-9ed729468381 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:16.492313 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ab910-66dc-4a6e-b0de-9ed729468381 None admin] PATH_INFO: `/v3/groups/f5e4c696c29b4f179b9957e5743c09b1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:16.493960 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:16.495955 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ab910-66dc-4a6e-b0de-9ed729468381 None admin] RBAC: Authorizing `identity:get_group(group_id=f5e4c696c29b4f179b9957e5743c09b1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:16.497464 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ab910-66dc-4a6e-b0de-9ed729468381 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:16.500022 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 183/371] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jun 23 19:48:16 2022] GET /identity/v3/groups/f5e4c696c29b4f179b9957e5743c09b1 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:16.500835 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ab910-66dc-4a6e-b0de-9ed729468381 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:16.544952 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a527f1-eb93-4eb2-bffa-bb61668a0f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:16.545817 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a527f1-eb93-4eb2-bffa-bb61668a0f69 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:16.546102 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a527f1-eb93-4eb2-bffa-bb61668a0f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:16.546379 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a527f1-eb93-4eb2-bffa-bb61668a0f69 None admin] PATH_INFO: `/v3/groups/f5e4c696c29b4f179b9957e5743c09b1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:16.548058 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a527f1-eb93-4eb2-bffa-bb61668a0f69 None admin] RBAC: Authorizing `identity:get_group(group_id=f5e4c696c29b4f179b9957e5743c09b1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:16.549673 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a527f1-eb93-4eb2-bffa-bb61668a0f69 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:16.556176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 189/372] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jun 23 19:48:16 2022] GET /identity/v3/groups/f5e4c696c29b4f179b9957e5743c09b1 => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:16.557080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a527f1-eb93-4eb2-bffa-bb61668a0f69 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:16.620709 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:16.637168 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ab910-66dc-4a6e-b0de-9ed729468381 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:16.671488 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a527f1-eb93-4eb2-bffa-bb61668a0f69 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:16.684595 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24790a67-5fab-4fda-be8c-71f276a0db90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:16.685818 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24790a67-5fab-4fda-be8c-71f276a0db90 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:16.686483 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24790a67-5fab-4fda-be8c-71f276a0db90 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:16.686929 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24790a67-5fab-4fda-be8c-71f276a0db90 None admin] PATH_INFO: `/v3/users/51d15ee3ffd24d7898ecc729628d7e96` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:16.689313 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24790a67-5fab-4fda-be8c-71f276a0db90 None admin] RBAC: Authorizing `identity:get_user(user_id=51d15ee3ffd24d7898ecc729628d7e96)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:16.691513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24790a67-5fab-4fda-be8c-71f276a0db90 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:16.693646 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 184/373] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:48:16 2022] GET /identity/v3/users/51d15ee3ffd24d7898ecc729628d7e96 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:16.695055 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24790a67-5fab-4fda-be8c-71f276a0db90 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:16.709568 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd50292-a79d-4edd-b55b-17031cc1369c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:16.710069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd50292-a79d-4edd-b55b-17031cc1369c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:16.710352 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd50292-a79d-4edd-b55b-17031cc1369c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:16.710626 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd50292-a79d-4edd-b55b-17031cc1369c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:16.759664 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:16.808555 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd50292-a79d-4edd-b55b-17031cc1369c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:16.811571 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd50292-a79d-4edd-b55b-17031cc1369c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:16.837117 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24790a67-5fab-4fda-be8c-71f276a0db90 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:16.874558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db5107-8862-4847-b1f2-a20b39efafc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:16.875213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db5107-8862-4847-b1f2-a20b39efafc5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:16.875576 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db5107-8862-4847-b1f2-a20b39efafc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:16.875774 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db5107-8862-4847-b1f2-a20b39efafc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:16.876642 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5db5107-8862-4847-b1f2-a20b39efafc5 None admin] RBAC: Adding query filter params (group.id=f5e4c696c29b4f179b9957e5743c09b1, role.id=03e4cfc753754643ae08fa909647cfe3, scope.project.id=4c94dfce039f4ff29bec2f9831ec1748) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:16.876956 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5db5107-8862-4847-b1f2-a20b39efafc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:16.878468 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5db5107-8862-4847-b1f2-a20b39efafc5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:16.885131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 185/374] 10.222.0.44 () {56 vars in 1376 bytes} [Thu Jun 23 19:48:16 2022] GET /identity/v3/role_assignments?group.id=f5e4c696c29b4f179b9957e5743c09b1&scope.project.id=4c94dfce039f4ff29bec2f9831ec1748&role.id=03e4cfc753754643ae08fa909647cfe3 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:16.886070 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db5107-8862-4847-b1f2-a20b39efafc5 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:16.910142 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:16.958216 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 190/375] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:48:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:48:16.959164 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd50292-a79d-4edd-b55b-17031cc1369c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:17.012658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db5107-8862-4847-b1f2-a20b39efafc5 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:17.047841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:17.051791 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84164391-ff37-44b9-84a0-7bfce4d55603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:17.052523 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84164391-ff37-44b9-84a0-7bfce4d55603 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:17.052852 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84164391-ff37-44b9-84a0-7bfce4d55603 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:17.053663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84164391-ff37-44b9-84a0-7bfce4d55603 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:17.054535 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84164391-ff37-44b9-84a0-7bfce4d55603 None admin] RBAC: Adding query filter params (group.id=f5e4c696c29b4f179b9957e5743c09b1, role.id=03e4cfc753754643ae08fa909647cfe3, scope.project.id=6c9c90447a664a9fa575e9c13ef585f0) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:17.054852 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84164391-ff37-44b9-84a0-7bfce4d55603 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:17.056595 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84164391-ff37-44b9-84a0-7bfce4d55603 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:17.063422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 186/376] 10.222.0.44 () {56 vars in 1376 bytes} [Thu Jun 23 19:48:16 2022] GET /identity/v3/role_assignments?group.id=f5e4c696c29b4f179b9957e5743c09b1&scope.project.id=6c9c90447a664a9fa575e9c13ef585f0&role.id=03e4cfc753754643ae08fa909647cfe3 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:17.064550 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84164391-ff37-44b9-84a0-7bfce4d55603 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:17.103205 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd50292-a79d-4edd-b55b-17031cc1369c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:17.174516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de96d39-3b22-419a-b299-a8cbd8763325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:17.175565 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de96d39-3b22-419a-b299-a8cbd8763325 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:17.176055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de96d39-3b22-419a-b299-a8cbd8763325 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:17.176591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de96d39-3b22-419a-b299-a8cbd8763325 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:17.177653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de96d39-3b22-419a-b299-a8cbd8763325 None admin] RBAC: Adding query filter params (role.id=43f9a6d53a0b43c6ada3d7d2eea1c7d2, user.id=51d15ee3ffd24d7898ecc729628d7e96, scope.system=all) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:17.178711 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de96d39-3b22-419a-b299-a8cbd8763325 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:17.180558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de96d39-3b22-419a-b299-a8cbd8763325 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:17.181215 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:17.202478 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84164391-ff37-44b9-84a0-7bfce4d55603 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:17.206763 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 191/377] 10.222.0.44 () {56 vars in 1308 bytes} [Thu Jun 23 19:48:16 2022] GET /identity/v3/role_assignments?user.id=51d15ee3ffd24d7898ecc729628d7e96&scope.system=all&role.id=43f9a6d53a0b43c6ada3d7d2eea1c7d2 => generated 510 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:17.246767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da385526-52df-44ba-9a64-d118cd573cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:17.247639 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da385526-52df-44ba-9a64-d118cd573cef None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:17.247924 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da385526-52df-44ba-9a64-d118cd573cef None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:17.248194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da385526-52df-44ba-9a64-d118cd573cef None admin] PATH_INFO: `/v3/projects/e3c8189cd37644e9a49e850123b33202` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:17.249446 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da385526-52df-44ba-9a64-d118cd573cef None admin] RBAC: Authorizing `identity:get_project(project_id=e3c8189cd37644e9a49e850123b33202)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:17.250953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da385526-52df-44ba-9a64-d118cd573cef None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:17.252969 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 187/378] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:48:17 2022] GET /identity/v3/projects/e3c8189cd37644e9a49e850123b33202 => generated 344 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:17.258932 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de96d39-3b22-419a-b299-a8cbd8763325 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:17.390612 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de96d39-3b22-419a-b299-a8cbd8763325 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:17.432037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:17.444363 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91310d-e396-4bfd-9bd1-878a31eea304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:17.445327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91310d-e396-4bfd-9bd1-878a31eea304 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:17.445610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91310d-e396-4bfd-9bd1-878a31eea304 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:17.447488 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91310d-e396-4bfd-9bd1-878a31eea304 None admin] PATH_INFO: `/v3/groups/f23cb458b0f2437d8574c068528dec1f` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:17.452452 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de91310d-e396-4bfd-9bd1-878a31eea304 None admin] RBAC: Authorizing `identity:get_group(group_id=f23cb458b0f2437d8574c068528dec1f)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:17.454027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de91310d-e396-4bfd-9bd1-878a31eea304 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:17.455697 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 192/379] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jun 23 19:48:17 2022] GET /identity/v3/groups/f23cb458b0f2437d8574c068528dec1f => generated 229 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:17.462180 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da385526-52df-44ba-9a64-d118cd573cef None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:17.567880 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da385526-52df-44ba-9a64-d118cd573cef None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:17.574771 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d281a-fd90-46f9-bb95-d994971404b4 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:17.575058 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d281a-fd90-46f9-bb95-d994971404b4 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:17.575329 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d281a-fd90-46f9-bb95-d994971404b4 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:17.577059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 193/380] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:48:17.582300 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0445bdf-90a1-4464-9591-c26803a3f5ef None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:17.582585 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0445bdf-90a1-4464-9591-c26803a3f5ef None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:17.582863 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0445bdf-90a1-4464-9591-c26803a3f5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:17.602365 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f0445bdf-90a1-4464-9591-c26803a3f5ef None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:17.603652 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f0445bdf-90a1-4464-9591-c26803a3f5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:17.618011 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57ba34-5f89-4ab1-97b0-05a87885d9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:17.618925 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57ba34-5f89-4ab1-97b0-05a87885d9f2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:17.619370 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57ba34-5f89-4ab1-97b0-05a87885d9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:17.619799 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57ba34-5f89-4ab1-97b0-05a87885d9f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:17.621152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57ba34-5f89-4ab1-97b0-05a87885d9f2 None admin] RBAC: Adding query filter params (group.id=f23cb458b0f2437d8574c068528dec1f, role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb, scope.project.id=e3c8189cd37644e9a49e850123b33202) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:17.621627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57ba34-5f89-4ab1-97b0-05a87885d9f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:17.624098 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57ba34-5f89-4ab1-97b0-05a87885d9f2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:17.634970 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 188/381] 10.222.0.44 () {56 vars in 1376 bytes} [Thu Jun 23 19:48:17 2022] GET /identity/v3/role_assignments?group.id=f23cb458b0f2437d8574c068528dec1f&scope.project.id=e3c8189cd37644e9a49e850123b33202&role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:17.724006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 194/382] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 19:48:17.751325 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57ba34-5f89-4ab1-97b0-05a87885d9f2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:17.774997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:17.878746 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57ba34-5f89-4ab1-97b0-05a87885d9f2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:17.917926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df57bc5-b72f-48ff-89b3-0b18d4dd7404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:17.918740 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df57bc5-b72f-48ff-89b3-0b18d4dd7404 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:17.919019 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df57bc5-b72f-48ff-89b3-0b18d4dd7404 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:17.919288 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df57bc5-b72f-48ff-89b3-0b18d4dd7404 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:17.930915 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df57bc5-b72f-48ff-89b3-0b18d4dd7404 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:17.942706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-0df57bc5-b72f-48ff-89b3-0b18d4dd7404 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:48:17.942706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:17.942706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:17.942706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:17.942706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:17.942706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:17.942706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:17.942706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:17.942706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:17.942706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:17.942706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:17.942706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:17.942706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:17.942706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:17.942706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:17.942706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:17.943663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:17.943663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:17.943663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:17.943663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:17.943663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:17.943663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:17.943663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:17.943663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:17.943663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:17.943663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:17.943663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:17.943663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:17.943663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:17.943663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:17.943663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:17.943663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:17.944513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:17.944513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:17.944513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:17.944513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:17.944513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:17.944513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:17.944513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:17.944513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:17.944513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:17.944513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:17.944513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:48:17.944513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:17.944513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 189/383] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:17.950262 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0445bdf-90a1-4464-9591-c26803a3f5ef None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:18.116676 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0445bdf-90a1-4464-9591-c26803a3f5ef None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:18.148654 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:18.159839 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b54a93-e149-4c07-a0e0-8816bb6b6df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:18.161613 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b54a93-e149-4c07-a0e0-8816bb6b6df1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:18.161613 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b54a93-e149-4c07-a0e0-8816bb6b6df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:18.161613 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b54a93-e149-4c07-a0e0-8816bb6b6df1 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:18.161929 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b54a93-e149-4c07-a0e0-8816bb6b6df1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:18.162172 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b54a93-e149-4c07-a0e0-8816bb6b6df1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:18.163595 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b54a93-e149-4c07-a0e0-8816bb6b6df1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:18.175775 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 195/384] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jun 23 19:48:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:18.181528 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df57bc5-b72f-48ff-89b3-0b18d4dd7404 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:18.289079 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:18.314932 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df57bc5-b72f-48ff-89b3-0b18d4dd7404 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:18.359904 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d5901-f5bf-4d5c-b204-b1fe6f609c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:18.360710 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d5901-f5bf-4d5c-b204-b1fe6f609c22 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:18.360994 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d5901-f5bf-4d5c-b204-b1fe6f609c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:18.361271 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d5901-f5bf-4d5c-b204-b1fe6f609c22 None admin] PATH_INFO: `/v3/groups/f5e4c696c29b4f179b9957e5743c09b1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:18.361449 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dace75-d641-4a6f-a6be-5331b1263664 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:18.361741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dace75-d641-4a6f-a6be-5331b1263664 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:18.362198 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dace75-d641-4a6f-a6be-5331b1263664 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:18.362478 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d5901-f5bf-4d5c-b204-b1fe6f609c22 None admin] RBAC: Authorizing `identity:get_group(group_id=f5e4c696c29b4f179b9957e5743c09b1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:18.363722 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 196/385] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:48:18.364007 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d5901-f5bf-4d5c-b204-b1fe6f609c22 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:18.366545 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 190/386] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jun 23 19:48:18 2022] GET /identity/v3/groups/f5e4c696c29b4f179b9957e5743c09b1 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:18.370117 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea77f07-fa11-4199-b527-f41c5642a984 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:18.370348 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea77f07-fa11-4199-b527-f41c5642a984 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:18.370625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea77f07-fa11-4199-b527-f41c5642a984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:18.371937 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d5901-f5bf-4d5c-b204-b1fe6f609c22 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:18.405438 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-cea77f07-fa11-4199-b527-f41c5642a984 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:18.406554 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-cea77f07-fa11-4199-b527-f41c5642a984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:18.492769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d5901-f5bf-4d5c-b204-b1fe6f609c22 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:18.541769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 197/387] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:18.543557 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ca90c-3ab6-4332-9199-36f6bc3fde62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:18.544487 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ca90c-3ab6-4332-9199-36f6bc3fde62 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:18.544771 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ca90c-3ab6-4332-9199-36f6bc3fde62 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:18.545040 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ca90c-3ab6-4332-9199-36f6bc3fde62 None admin] PATH_INFO: `/v3/projects/4c94dfce039f4ff29bec2f9831ec1748` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:18.546254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7ca90c-3ab6-4332-9199-36f6bc3fde62 None admin] RBAC: Authorizing `identity:get_project(project_id=4c94dfce039f4ff29bec2f9831ec1748)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:18.547768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7ca90c-3ab6-4332-9199-36f6bc3fde62 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:18.549895 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 191/388] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:48:18 2022] GET /identity/v3/projects/4c94dfce039f4ff29bec2f9831ec1748 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:18.555851 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea77f07-fa11-4199-b527-f41c5642a984 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:18.571976 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ca90c-3ab6-4332-9199-36f6bc3fde62 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:18.596325 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:18.708449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ca90c-3ab6-4332-9199-36f6bc3fde62 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:18.737200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea77f07-fa11-4199-b527-f41c5642a984 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:18.766038 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:18.786469 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa0403-76bc-40e9-8542-0dc13c9972cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:18.787381 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa0403-76bc-40e9-8542-0dc13c9972cf None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:18.787659 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa0403-76bc-40e9-8542-0dc13c9972cf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:18.787932 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa0403-76bc-40e9-8542-0dc13c9972cf None admin] PATH_INFO: `/v3/projects/4c94dfce039f4ff29bec2f9831ec1748/groups/f5e4c696c29b4f179b9957e5743c09b1/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:18.790787 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d3842-f97c-418f-b12c-2ee657bc293d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:18.791586 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d3842-f97c-418f-b12c-2ee657bc293d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:18.791863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d3842-f97c-418f-b12c-2ee657bc293d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:18.792130 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d3842-f97c-418f-b12c-2ee657bc293d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:18.793168 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa0403-76bc-40e9-8542-0dc13c9972cf None admin] RBAC: Authorizing `identity:create_grant(project_id=4c94dfce039f4ff29bec2f9831ec1748, group_id=f5e4c696c29b4f179b9957e5743c09b1, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:18.794602 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa0403-76bc-40e9-8542-0dc13c9972cf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:18.809587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053d3842-f97c-418f-b12c-2ee657bc293d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:18.817687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-053d3842-f97c-418f-b12c-2ee657bc293d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:48:18.817687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:18.817687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:18.817687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:18.817687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:18.817687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:18.817687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:18.817687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:18.817687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:18.817687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:18.817687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:18.817687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:18.817687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:18.817687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:18.817687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:18.817687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:18.819587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:18.819587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:18.819587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:18.819587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:18.819587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:18.819587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:18.819587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:18.819587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:18.819587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:18.819587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:18.819587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:18.819587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:18.819587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:18.819587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:18.819587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:18.819587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:18.820423 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:18.820423 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:18.820423 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:18.820423 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:18.820423 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:18.820423 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:18.820423 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:18.820423 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:18.820423 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:18.820423 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:18.820423 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:48:18.820423 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:18.820423 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 192/389] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:48:18.826189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d3842-f97c-418f-b12c-2ee657bc293d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:18.885700 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 198/390] 10.222.0.44 () {58 vars in 1442 bytes} [Thu Jun 23 19:48:18 2022] PUT /identity/v3/projects/4c94dfce039f4ff29bec2f9831ec1748/groups/f5e4c696c29b4f179b9957e5743c09b1/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 19:48:18.908532 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:18.943031 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d3842-f97c-418f-b12c-2ee657bc293d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:18.979837 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22603b4d-a973-4a5c-ae5c-39a1d6df8ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:18.980481 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22603b4d-a973-4a5c-ae5c-39a1d6df8ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:18.980768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22603b4d-a973-4a5c-ae5c-39a1d6df8ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:18.981040 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22603b4d-a973-4a5c-ae5c-39a1d6df8ec4 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:18.981790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22603b4d-a973-4a5c-ae5c-39a1d6df8ec4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:18.982093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22603b4d-a973-4a5c-ae5c-39a1d6df8ec4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:18.983490 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22603b4d-a973-4a5c-ae5c-39a1d6df8ec4 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:18.993614 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 193/391] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jun 23 19:48:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:18.999243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa0403-76bc-40e9-8542-0dc13c9972cf None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:19.110084 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa0403-76bc-40e9-8542-0dc13c9972cf None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:19.151854 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afc071e-c33e-4a02-946d-ae1e7c252618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:19.152770 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc071e-c33e-4a02-946d-ae1e7c252618 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:19.153053 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc071e-c33e-4a02-946d-ae1e7c252618 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:19.153320 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc071e-c33e-4a02-946d-ae1e7c252618 None admin] PATH_INFO: `/v3/groups/f5e4c696c29b4f179b9957e5743c09b1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:19.154542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afc071e-c33e-4a02-946d-ae1e7c252618 None admin] RBAC: Authorizing `identity:get_group(group_id=f5e4c696c29b4f179b9957e5743c09b1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:19.155981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afc071e-c33e-4a02-946d-ae1e7c252618 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:19.157971 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 199/392] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jun 23 19:48:18 2022] GET /identity/v3/groups/f5e4c696c29b4f179b9957e5743c09b1 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:19.163263 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22603b4d-a973-4a5c-ae5c-39a1d6df8ec4 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:19.276635 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22603b4d-a973-4a5c-ae5c-39a1d6df8ec4 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:19.311363 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a340bc-a100-4680-aa00-afd23f7e6e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:19.312230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a340bc-a100-4680-aa00-afd23f7e6e7d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:19.312533 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a340bc-a100-4680-aa00-afd23f7e6e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:19.313062 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a340bc-a100-4680-aa00-afd23f7e6e7d None admin] PATH_INFO: `/v3/projects/6c9c90447a664a9fa575e9c13ef585f0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:19.314249 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a340bc-a100-4680-aa00-afd23f7e6e7d None admin] RBAC: Authorizing `identity:get_project(project_id=6c9c90447a664a9fa575e9c13ef585f0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:19.315713 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a340bc-a100-4680-aa00-afd23f7e6e7d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:19.317716 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 194/393] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:48:19 2022] GET /identity/v3/projects/6c9c90447a664a9fa575e9c13ef585f0 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:19.323376 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afc071e-c33e-4a02-946d-ae1e7c252618 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:19.383227 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:19.453514 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afc071e-c33e-4a02-946d-ae1e7c252618 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:19.504394 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7711e-9cae-41b0-925b-9747e2bca692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:19.505214 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7711e-9cae-41b0-925b-9747e2bca692 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:19.505498 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7711e-9cae-41b0-925b-9747e2bca692 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:19.505767 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7711e-9cae-41b0-925b-9747e2bca692 None admin] PATH_INFO: `/v3/projects/6c9c90447a664a9fa575e9c13ef585f0/groups/f5e4c696c29b4f179b9957e5743c09b1/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:19.507798 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c7711e-9cae-41b0-925b-9747e2bca692 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c9c90447a664a9fa575e9c13ef585f0, group_id=f5e4c696c29b4f179b9957e5743c09b1, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:19.509339 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c7711e-9cae-41b0-925b-9747e2bca692 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:19.531772 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c8145-379e-4fed-a052-d15d92920bc7 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:19.532068 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c8145-379e-4fed-a052-d15d92920bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:19.532348 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c8145-379e-4fed-a052-d15d92920bc7 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:19.534040 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 195/394] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:19.539251 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd69d6-fce6-4f35-bfdf-9f984b30cebb None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:19.539537 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd69d6-fce6-4f35-bfdf-9f984b30cebb None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:19.539808 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd69d6-fce6-4f35-bfdf-9f984b30cebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:19.545028 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:19.557565 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d7bd69d6-fce6-4f35-bfdf-9f984b30cebb None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:19.558803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d7bd69d6-fce6-4f35-bfdf-9f984b30cebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:19.597487 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 200/395] 10.222.0.44 () {58 vars in 1442 bytes} [Thu Jun 23 19:48:19 2022] PUT /identity/v3/projects/6c9c90447a664a9fa575e9c13ef585f0/groups/f5e4c696c29b4f179b9957e5743c09b1/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:48:19.651620 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 196/396] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 19:48:19.675529 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7711e-9cae-41b0-925b-9747e2bca692 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:19.814823 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7711e-9cae-41b0-925b-9747e2bca692 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:19.821482 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:19.864890 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a3fe3-06db-4745-8d3d-4c17e53e50e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:19.866607 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a3fe3-06db-4745-8d3d-4c17e53e50e2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:19.867094 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a3fe3-06db-4745-8d3d-4c17e53e50e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:19.867566 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a3fe3-06db-4745-8d3d-4c17e53e50e2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:19.883809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a3fe3-06db-4745-8d3d-4c17e53e50e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:19.892219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-fe9a3fe3-06db-4745-8d3d-4c17e53e50e2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 19:48:19.892219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:19.892219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:19.892219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:19.892219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:19.892219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:19.892219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:19.892219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:19.892219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:19.892219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:19.892219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:19.892219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:19.892219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:19.892219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:19.892219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:19.892219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:19.893120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:19.893120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:19.893120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:19.893120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:19.893120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:19.893120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:19.893120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:19.893120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:19.893120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:19.893120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:19.893120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:19.893120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:19.893120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:19.893120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:19.893120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:19.893120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:19.893937 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:19.893937 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:19.893937 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:19.893937 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:19.893937 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:19.893937 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:19.893937 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:19.893937 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:19.893937 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:19.893937 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:19.893937 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 19:48:19.893937 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:19.896694 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 201/397] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jun 23 19:48:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:48:19.904336 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd69d6-fce6-4f35-bfdf-9f984b30cebb None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:20.024760 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd69d6-fce6-4f35-bfdf-9f984b30cebb None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:20.055705 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:20.060559 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed008704-f81f-4b14-88c6-d988e4e99266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:20.061228 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed008704-f81f-4b14-88c6-d988e4e99266 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:20.061505 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed008704-f81f-4b14-88c6-d988e4e99266 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:20.061830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed008704-f81f-4b14-88c6-d988e4e99266 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:20.062583 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed008704-f81f-4b14-88c6-d988e4e99266 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:20.063233 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed008704-f81f-4b14-88c6-d988e4e99266 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:20.064640 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed008704-f81f-4b14-88c6-d988e4e99266 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:20.074060 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 197/398] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:48:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:20.079221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a3fe3-06db-4745-8d3d-4c17e53e50e2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:20.215452 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f405a-96e4-49c1-b1cd-106e18ce77fd None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:20.215774 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f405a-96e4-49c1-b1cd-106e18ce77fd None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:20.216013 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f405a-96e4-49c1-b1cd-106e18ce77fd None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:20.217655 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 198/399] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:20.232244 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0a657-6348-444b-b93a-6940a69cdf50 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:20.232576 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0a657-6348-444b-b93a-6940a69cdf50 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:20.232845 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0a657-6348-444b-b93a-6940a69cdf50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:20.237221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a3fe3-06db-4745-8d3d-4c17e53e50e2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:20.269082 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3ff0a657-6348-444b-b93a-6940a69cdf50 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:20.270274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3ff0a657-6348-444b-b93a-6940a69cdf50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:20.274107 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96fb89-1357-4d10-b87d-83b9d8847f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:20.275030 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96fb89-1357-4d10-b87d-83b9d8847f57 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:20.275320 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96fb89-1357-4d10-b87d-83b9d8847f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:20.275650 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96fb89-1357-4d10-b87d-83b9d8847f57 None admin] PATH_INFO: `/v3/groups/f5e4c696c29b4f179b9957e5743c09b1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:20.276914 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c96fb89-1357-4d10-b87d-83b9d8847f57 None admin] RBAC: Authorizing `identity:get_group(group_id=f5e4c696c29b4f179b9957e5743c09b1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:20.278381 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c96fb89-1357-4d10-b87d-83b9d8847f57 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:20.280382 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 202/400] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jun 23 19:48:20 2022] GET /identity/v3/groups/f5e4c696c29b4f179b9957e5743c09b1 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:20.285232 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96fb89-1357-4d10-b87d-83b9d8847f57 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:20.318907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:20.362178 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 199/401] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:20.363648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e54d69b-a5d1-42c4-9505-6f3f168f693b None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:20.363760 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e54d69b-a5d1-42c4-9505-6f3f168f693b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:20.364027 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e54d69b-a5d1-42c4-9505-6f3f168f693b None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:20.369606 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 200/402] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:20.371015 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043203b-1f87-4407-a018-dd6535db30a2 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:20.371144 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043203b-1f87-4407-a018-dd6535db30a2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:20.371330 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043203b-1f87-4407-a018-dd6535db30a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:20.389291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-2043203b-1f87-4407-a018-dd6535db30a2 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:20.390827 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2043203b-1f87-4407-a018-dd6535db30a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:20.397033 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96fb89-1357-4d10-b87d-83b9d8847f57 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:20.435963 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b536aa71-d1b7-4d19-ba62-2a567ab6b32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:20.436894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536aa71-d1b7-4d19-ba62-2a567ab6b32c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:20.437172 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536aa71-d1b7-4d19-ba62-2a567ab6b32c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:20.437443 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536aa71-d1b7-4d19-ba62-2a567ab6b32c None admin] PATH_INFO: `/v3/projects/4c94dfce039f4ff29bec2f9831ec1748` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:20.439346 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b536aa71-d1b7-4d19-ba62-2a567ab6b32c None admin] RBAC: Authorizing `identity:get_project(project_id=4c94dfce039f4ff29bec2f9831ec1748)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:20.441243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b536aa71-d1b7-4d19-ba62-2a567ab6b32c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:20.447933 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 203/403] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:48:20 2022] GET /identity/v3/projects/4c94dfce039f4ff29bec2f9831ec1748 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:20.449948 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b536aa71-d1b7-4d19-ba62-2a567ab6b32c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:20.480322 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 201/404] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:20.480322 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043203b-1f87-4407-a018-dd6535db30a2 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:20.564495 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:20.585779 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:20.588700 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b536aa71-d1b7-4d19-ba62-2a567ab6b32c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:20.624698 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b238b-df96-49cc-91bd-747dc4c64b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:20.625825 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b238b-df96-49cc-91bd-747dc4c64b13 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:20.626332 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b238b-df96-49cc-91bd-747dc4c64b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:20.626805 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b238b-df96-49cc-91bd-747dc4c64b13 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:20.628526 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043203b-1f87-4407-a018-dd6535db30a2 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:20.637624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b238b-df96-49cc-91bd-747dc4c64b13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:20.646370 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-395b238b-df96-49cc-91bd-747dc4c64b13 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 19:48:20.646370 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:20.646370 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:20.646370 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:20.646370 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:20.646370 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:20.646370 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:20.646370 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:20.646370 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:20.646370 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:20.646370 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:20.646370 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:20.646370 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:20.646370 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:20.646370 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:20.646370 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:20.647838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:20.647838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:20.647838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:20.647838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:20.647838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:20.647838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:20.647838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:20.647838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:20.647838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:20.647838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:20.647838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:20.647838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:20.647838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:20.647838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:20.647838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:20.647838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:20.648647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:20.648647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:20.648647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:20.648647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:20.648647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:20.648647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:20.648647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:20.648647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:20.648647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:20.648647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:20.648647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 19:48:20.648647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:20.651301 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 204/405] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jun 23 19:48:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:20.652401 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b238b-df96-49cc-91bd-747dc4c64b13 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:20.670925 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb41c6-805b-4649-bdb6-97344afd0c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:20.672088 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb41c6-805b-4649-bdb6-97344afd0c0b None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:20.672839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb41c6-805b-4649-bdb6-97344afd0c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:20.673537 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb41c6-805b-4649-bdb6-97344afd0c0b None admin] PATH_INFO: `/v3/projects/4c94dfce039f4ff29bec2f9831ec1748/groups/f5e4c696c29b4f179b9957e5743c09b1/roles/03e4cfc753754643ae08fa909647cfe3` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:20.677131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb41c6-805b-4649-bdb6-97344afd0c0b None admin] RBAC: Authorizing `identity:create_grant(project_id=4c94dfce039f4ff29bec2f9831ec1748, group_id=f5e4c696c29b4f179b9957e5743c09b1, role_id=03e4cfc753754643ae08fa909647cfe3)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:20.678899 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb41c6-805b-4649-bdb6-97344afd0c0b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:20.757836 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b238b-df96-49cc-91bd-747dc4c64b13 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:20.780937 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 202/406] 10.222.0.44 () {58 vars in 1442 bytes} [Thu Jun 23 19:48:20 2022] PUT /identity/v3/projects/4c94dfce039f4ff29bec2f9831ec1748/groups/f5e4c696c29b4f179b9957e5743c09b1/roles/03e4cfc753754643ae08fa909647cfe3 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:48:20.782708 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb41c6-805b-4649-bdb6-97344afd0c0b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:20.813356 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5421c-3570-406a-8fc4-aa019e81e5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:20.814176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5421c-3570-406a-8fc4-aa019e81e5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:20.816083 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5421c-3570-406a-8fc4-aa019e81e5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:20.816460 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f5421c-3570-406a-8fc4-aa019e81e5d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:20.841970 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f5421c-3570-406a-8fc4-aa019e81e5d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:20.855734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-35f5421c-3570-406a-8fc4-aa019e81e5d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:48:20.855734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:20.855734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:20.855734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:20.855734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:20.855734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:20.855734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:20.855734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:20.855734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:20.855734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:20.855734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:20.855734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:20.855734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:20.855734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:20.855734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:20.855734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:20.856732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:20.856732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:20.856732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:20.856732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:20.856732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:20.856732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:20.856732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:20.856732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:20.856732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:20.856732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:20.856732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:20.856732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:20.856732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:20.856732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:20.856732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:20.856732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:20.857525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:20.857525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:20.857525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:20.857525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:20.857525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:20.857525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:20.857525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:20.857525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:20.857525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:20.857525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:20.857525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:48:20.857525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:20.859681 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 205/407] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:48:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:20.867057 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5421c-3570-406a-8fc4-aa019e81e5d6 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:20.920348 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:20.954065 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb41c6-805b-4649-bdb6-97344afd0c0b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:20.973502 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:20.999258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4de763-89fc-4bc8-a815-5e54c648beb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:21.000176 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4de763-89fc-4bc8-a815-5e54c648beb5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:21.000834 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4de763-89fc-4bc8-a815-5e54c648beb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:21.001357 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4de763-89fc-4bc8-a815-5e54c648beb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:21.002255 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4de763-89fc-4bc8-a815-5e54c648beb5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:21.002678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4de763-89fc-4bc8-a815-5e54c648beb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:21.004563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4de763-89fc-4bc8-a815-5e54c648beb5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:21.016947 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 203/408] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:48:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:21.025353 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4de763-89fc-4bc8-a815-5e54c648beb5 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:21.078721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f5421c-3570-406a-8fc4-aa019e81e5d6 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:21.127647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99df0a6-54fb-4a05-9d05-59314b9ff33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:21.128940 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99df0a6-54fb-4a05-9d05-59314b9ff33c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:21.129057 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99df0a6-54fb-4a05-9d05-59314b9ff33c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:21.129313 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99df0a6-54fb-4a05-9d05-59314b9ff33c None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:21.130071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99df0a6-54fb-4a05-9d05-59314b9ff33c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:21.130380 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99df0a6-54fb-4a05-9d05-59314b9ff33c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:21.133001 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99df0a6-54fb-4a05-9d05-59314b9ff33c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:21.144821 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 206/409] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:21.153350 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99df0a6-54fb-4a05-9d05-59314b9ff33c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:21.161465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4de763-89fc-4bc8-a815-5e54c648beb5 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:21.198844 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6b019-60c3-4f41-a478-9bf787b9b853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:21.199966 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6b019-60c3-4f41-a478-9bf787b9b853 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:21.200574 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6b019-60c3-4f41-a478-9bf787b9b853 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:21.200992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6b019-60c3-4f41-a478-9bf787b9b853 None admin] PATH_INFO: `/v3/groups/f5e4c696c29b4f179b9957e5743c09b1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:21.202380 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d6b019-60c3-4f41-a478-9bf787b9b853 None admin] RBAC: Authorizing `identity:get_group(group_id=f5e4c696c29b4f179b9957e5743c09b1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:21.204164 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d6b019-60c3-4f41-a478-9bf787b9b853 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:21.206378 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 204/410] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jun 23 19:48:21 2022] GET /identity/v3/groups/f5e4c696c29b4f179b9957e5743c09b1 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:21.211426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6b019-60c3-4f41-a478-9bf787b9b853 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:21.277060 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:21.349522 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:21.355014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6b019-60c3-4f41-a478-9bf787b9b853 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:21.371197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99df0a6-54fb-4a05-9d05-59314b9ff33c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:21.398704 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9b5dd-a82e-43ed-baed-406b351866d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:21.399703 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9b5dd-a82e-43ed-baed-406b351866d1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:21.399986 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9b5dd-a82e-43ed-baed-406b351866d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:21.400261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9b5dd-a82e-43ed-baed-406b351866d1 None admin] PATH_INFO: `/v3/projects/6c9c90447a664a9fa575e9c13ef585f0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:21.401604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9b5dd-a82e-43ed-baed-406b351866d1 None admin] RBAC: Authorizing `identity:get_project(project_id=6c9c90447a664a9fa575e9c13ef585f0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:21.403167 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9b5dd-a82e-43ed-baed-406b351866d1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:21.405565 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 205/411] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:48:21 2022] GET /identity/v3/projects/6c9c90447a664a9fa575e9c13ef585f0 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:21.406999 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c71798-9fec-46e4-80fc-ac96b67ca08b None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:21.407289 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c71798-9fec-46e4-80fc-ac96b67ca08b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:21.407564 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c71798-9fec-46e4-80fc-ac96b67ca08b None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:21.407686 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d2638-896c-4c2a-8b69-cf3b00edec70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:21.408698 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d2638-896c-4c2a-8b69-cf3b00edec70 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:21.408829 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 206/412] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:21.409156 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d2638-896c-4c2a-8b69-cf3b00edec70 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:21.409485 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d2638-896c-4c2a-8b69-cf3b00edec70 None admin] PATH_INFO: `/v3/groups/f23cb458b0f2437d8574c068528dec1f` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:21.410718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d2638-896c-4c2a-8b69-cf3b00edec70 None admin] RBAC: Authorizing `identity:get_group(group_id=f23cb458b0f2437d8574c068528dec1f)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:21.412263 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d2638-896c-4c2a-8b69-cf3b00edec70 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:21.414558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 207/413] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jun 23 19:48:21 2022] GET /identity/v3/groups/f23cb458b0f2437d8574c068528dec1f => generated 229 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:21.419320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18530a64-d10f-47c6-82ab-d4a686898691 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:21.419320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18530a64-d10f-47c6-82ab-d4a686898691 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:21.419320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18530a64-d10f-47c6-82ab-d4a686898691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:21.420141 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d2638-896c-4c2a-8b69-cf3b00edec70 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:21.441728 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-18530a64-d10f-47c6-82ab-d4a686898691 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:21.443506 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-18530a64-d10f-47c6-82ab-d4a686898691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:21.520571 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 207/414] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:21.522347 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18530a64-d10f-47c6-82ab-d4a686898691 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:21.538599 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d2638-896c-4c2a-8b69-cf3b00edec70 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:21.566958 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:21.576260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff71834-2cac-4ccf-9972-db128c5dd68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:21.583464 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff71834-2cac-4ccf-9972-db128c5dd68b None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:21.583464 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff71834-2cac-4ccf-9972-db128c5dd68b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:21.583464 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff71834-2cac-4ccf-9972-db128c5dd68b None admin] PATH_INFO: `/v3/projects/6c9c90447a664a9fa575e9c13ef585f0/groups/f5e4c696c29b4f179b9957e5743c09b1/roles/03e4cfc753754643ae08fa909647cfe3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:21.583464 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff71834-2cac-4ccf-9972-db128c5dd68b None admin] RBAC: Authorizing `identity:create_grant(project_id=6c9c90447a664a9fa575e9c13ef585f0, group_id=f5e4c696c29b4f179b9957e5743c09b1, role_id=03e4cfc753754643ae08fa909647cfe3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:21.583464 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff71834-2cac-4ccf-9972-db128c5dd68b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:21.627349 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18530a64-d10f-47c6-82ab-d4a686898691 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:21.663736 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d5712-7513-4459-a50a-1e10b18729f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:21.664694 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d5712-7513-4459-a50a-1e10b18729f6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:21.664974 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d5712-7513-4459-a50a-1e10b18729f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:21.665239 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d5712-7513-4459-a50a-1e10b18729f6 None admin] PATH_INFO: `/v3/projects/e3c8189cd37644e9a49e850123b33202` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:21.666418 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d5712-7513-4459-a50a-1e10b18729f6 None admin] RBAC: Authorizing `identity:get_project(project_id=e3c8189cd37644e9a49e850123b33202)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:21.667875 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d5712-7513-4459-a50a-1e10b18729f6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:21.669768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 208/415] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:48:21 2022] GET /identity/v3/projects/e3c8189cd37644e9a49e850123b33202 => generated 344 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:21.674713 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d5712-7513-4459-a50a-1e10b18729f6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:21.694591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 208/416] 10.222.0.44 () {58 vars in 1442 bytes} [Thu Jun 23 19:48:21 2022] PUT /identity/v3/projects/6c9c90447a664a9fa575e9c13ef585f0/groups/f5e4c696c29b4f179b9957e5743c09b1/roles/03e4cfc753754643ae08fa909647cfe3 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:48:21.696691 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff71834-2cac-4ccf-9972-db128c5dd68b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:21.713636 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:21.840348 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff71834-2cac-4ccf-9972-db128c5dd68b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:21.882973 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e92f409-c657-4c6b-ba05-f6987401e494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:21.884073 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92f409-c657-4c6b-ba05-f6987401e494 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:21.884493 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92f409-c657-4c6b-ba05-f6987401e494 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:21.884876 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e92f409-c657-4c6b-ba05-f6987401e494 None admin] PATH_INFO: `/v3/projects/e3c8189cd37644e9a49e850123b33202/groups/f23cb458b0f2437d8574c068528dec1f/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:21.887726 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e92f409-c657-4c6b-ba05-f6987401e494 None admin] RBAC: Authorizing `identity:create_grant(project_id=e3c8189cd37644e9a49e850123b33202, group_id=f23cb458b0f2437d8574c068528dec1f, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:21.889673 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e92f409-c657-4c6b-ba05-f6987401e494 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:21.917707 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d5712-7513-4459-a50a-1e10b18729f6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:21.931939 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:21.957012 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2a82a-6a10-4227-a936-e64d09e2cc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:21.957902 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2a82a-6a10-4227-a936-e64d09e2cc73 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:21.958188 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2a82a-6a10-4227-a936-e64d09e2cc73 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:21.958462 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2a82a-6a10-4227-a936-e64d09e2cc73 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:21.969719 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a2a82a-6a10-4227-a936-e64d09e2cc73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:21.973572 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 209/417] 10.222.0.44 () {58 vars in 1442 bytes} [Thu Jun 23 19:48:21 2022] PUT /identity/v3/projects/e3c8189cd37644e9a49e850123b33202/groups/f23cb458b0f2437d8574c068528dec1f/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:48:21.975469 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd47b7-3b3e-4e00-9268-929b86876bea None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:21.975835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd47b7-3b3e-4e00-9268-929b86876bea None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:21.976264 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd47b7-3b3e-4e00-9268-929b86876bea None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:21.978036 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 210/418] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:21.978372 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-07a2a82a-6a10-4227-a936-e64d09e2cc73 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:48:21.978372 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:21.978372 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:21.978372 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:21.978372 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:21.978372 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:21.978372 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:21.978372 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:21.978372 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:21.978372 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:21.978372 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:21.978372 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:21.978372 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:21.978372 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:21.978372 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:21.978372 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:21.979230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:21.979230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:21.979230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:21.979230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:21.979230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:21.979230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:21.979230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:21.979230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:21.979230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:21.979230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:21.979230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:21.979230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:21.979230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:21.979230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:21.979230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:21.979230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:21.980119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:21.980119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:21.980119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:21.980119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:21.980119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:21.980119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:21.980119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:21.980119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:21.980119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:21.980119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:21.980119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:48:21.980119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:21.984437 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 209/419] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:21.985715 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd47b7-3b3e-4e00-9268-929b86876bea None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:21.990962 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77220be-28eb-4c2e-a3e6-84d3be04c76c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:21.991381 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77220be-28eb-4c2e-a3e6-84d3be04c76c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:21.991812 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77220be-28eb-4c2e-a3e6-84d3be04c76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:22.026694 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d77220be-28eb-4c2e-a3e6-84d3be04c76c None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:22.030992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d77220be-28eb-4c2e-a3e6-84d3be04c76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:22.123280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:22.153748 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 210/420] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:22.164065 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd47b7-3b3e-4e00-9268-929b86876bea None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:22.184815 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77220be-28eb-4c2e-a3e6-84d3be04c76c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:22.206688 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64c2cb9-4049-4a12-9fd1-cd10a117d876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:22.207806 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c2cb9-4049-4a12-9fd1-cd10a117d876 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:22.208275 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c2cb9-4049-4a12-9fd1-cd10a117d876 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:22.208719 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c2cb9-4049-4a12-9fd1-cd10a117d876 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:22.210360 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64c2cb9-4049-4a12-9fd1-cd10a117d876 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:22.210807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64c2cb9-4049-4a12-9fd1-cd10a117d876 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:22.212805 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64c2cb9-4049-4a12-9fd1-cd10a117d876 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:22.222434 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 211/421] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jun 23 19:48:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:22.233505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64c2cb9-4049-4a12-9fd1-cd10a117d876 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:22.291017 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:22.293254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77220be-28eb-4c2e-a3e6-84d3be04c76c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:22.328759 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5bd88-191b-49ce-8e4e-f72cd079041e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:22.329570 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5bd88-191b-49ce-8e4e-f72cd079041e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:22.330175 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5bd88-191b-49ce-8e4e-f72cd079041e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:22.330464 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5bd88-191b-49ce-8e4e-f72cd079041e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:22.342886 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5bd88-191b-49ce-8e4e-f72cd079041e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:22.352290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-33f5bd88-191b-49ce-8e4e-f72cd079041e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:48:22.352290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:22.352290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:22.352290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:22.352290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:22.352290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:22.352290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:22.352290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:22.352290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:22.352290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:22.352290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:22.352290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:22.352290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:22.352290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:22.352290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:22.352290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:22.362807 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:22.362807 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:22.362807 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:22.362807 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:22.362807 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:22.362807 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:22.362807 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:22.362807 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:22.362807 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:22.362807 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:22.362807 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:22.362807 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:22.362807 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:22.362807 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:22.362807 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:22.362807 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:22.363795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:22.363795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:22.363795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:22.363795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:22.363795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:22.363795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:22.363795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:22.363795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:22.363795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:22.363795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:22.363795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 19:48:22.363795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:22.363795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 211/422] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:48:22.363795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5bd88-191b-49ce-8e4e-f72cd079041e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:22.428765 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64c2cb9-4049-4a12-9fd1-cd10a117d876 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:22.497649 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5bd88-191b-49ce-8e4e-f72cd079041e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:22.533770 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:22.534161 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a0e56-24c8-492f-be02-9f9c174bd22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:22.534919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a0e56-24c8-492f-be02-9f9c174bd22e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:22.535271 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a0e56-24c8-492f-be02-9f9c174bd22e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:22.535619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a0e56-24c8-492f-be02-9f9c174bd22e None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:22.536480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a0e56-24c8-492f-be02-9f9c174bd22e None admin] RBAC: Adding query filter params (name=member) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:22.536857 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a0e56-24c8-492f-be02-9f9c174bd22e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:22.538945 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a0e56-24c8-492f-be02-9f9c174bd22e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:22.547852 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 212/423] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jun 23 19:48:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:22.553284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a0e56-24c8-492f-be02-9f9c174bd22e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:22.556177 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c9d208-8a11-45f9-8977-d2b75f394a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:22.556871 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9d208-8a11-45f9-8977-d2b75f394a82 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:22.557208 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9d208-8a11-45f9-8977-d2b75f394a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:22.557481 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c9d208-8a11-45f9-8977-d2b75f394a82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:22.580554 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:22.661505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c9d208-8a11-45f9-8977-d2b75f394a82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:22.663809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c9d208-8a11-45f9-8977-d2b75f394a82 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:22.774552 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a0e56-24c8-492f-be02-9f9c174bd22e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:22.779297 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 212/424] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:48:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:48:22.784561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c9d208-8a11-45f9-8977-d2b75f394a82 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:22.819145 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:22.889562 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c542cba-27ab-4efe-9226-23ddb519c1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:22.890219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c542cba-27ab-4efe-9226-23ddb519c1ec None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:22.890501 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c542cba-27ab-4efe-9226-23ddb519c1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:22.890773 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c542cba-27ab-4efe-9226-23ddb519c1ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:22.898692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c9d208-8a11-45f9-8977-d2b75f394a82 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:22.938282 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887cd1b7-cd20-418b-b049-03829437a2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:22.939485 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cd1b7-cd20-418b-b049-03829437a2dd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:22.939988 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cd1b7-cd20-418b-b049-03829437a2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:22.940655 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cd1b7-cd20-418b-b049-03829437a2dd None admin] PATH_INFO: `/v3/projects/4c94dfce039f4ff29bec2f9831ec1748` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:22.942295 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887cd1b7-cd20-418b-b049-03829437a2dd None admin] RBAC: Authorizing `identity:get_project(project_id=4c94dfce039f4ff29bec2f9831ec1748)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:22.944593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887cd1b7-cd20-418b-b049-03829437a2dd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:22.947324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 213/425] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:48:22 2022] GET /identity/v3/projects/4c94dfce039f4ff29bec2f9831ec1748 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:22.952935 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887cd1b7-cd20-418b-b049-03829437a2dd None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:23.057258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:23.060819 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887cd1b7-cd20-418b-b049-03829437a2dd None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:23.077411 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c542cba-27ab-4efe-9226-23ddb519c1ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:23.078919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c542cba-27ab-4efe-9226-23ddb519c1ec None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:23.107190 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde8218-156e-409c-9762-d326c8e6b4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:23.109235 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde8218-156e-409c-9762-d326c8e6b4bb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:23.109934 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde8218-156e-409c-9762-d326c8e6b4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:23.111091 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde8218-156e-409c-9762-d326c8e6b4bb None admin] PATH_INFO: `/v3/groups/f5e4c696c29b4f179b9957e5743c09b1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:23.113816 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dde8218-156e-409c-9762-d326c8e6b4bb None admin] RBAC: Authorizing `identity:get_group(group_id=f5e4c696c29b4f179b9957e5743c09b1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:23.117520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dde8218-156e-409c-9762-d326c8e6b4bb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:23.121305 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 214/426] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jun 23 19:48:22 2022] GET /identity/v3/groups/f5e4c696c29b4f179b9957e5743c09b1 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:23.127199 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde8218-156e-409c-9762-d326c8e6b4bb None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:23.136874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:23.279489 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 213/427] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:48:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 727 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 19:48:23.289406 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c542cba-27ab-4efe-9226-23ddb519c1ec None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:23.310845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde8218-156e-409c-9762-d326c8e6b4bb None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:23.377640 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88c351-dd71-4cc4-a912-d07bf4070498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:23.378348 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88c351-dd71-4cc4-a912-d07bf4070498 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:23.378632 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88c351-dd71-4cc4-a912-d07bf4070498 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:23.378900 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88c351-dd71-4cc4-a912-d07bf4070498 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:23.379732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88c351-dd71-4cc4-a912-d07bf4070498 None admin] RBAC: Adding query filter params (group.id=f5e4c696c29b4f179b9957e5743c09b1, role.id=860491bfbb574772a7a213d6a25962e0, scope.project.id=4c94dfce039f4ff29bec2f9831ec1748) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:23.380045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88c351-dd71-4cc4-a912-d07bf4070498 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:23.381971 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88c351-dd71-4cc4-a912-d07bf4070498 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:23.392718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 215/428] 10.222.0.44 () {56 vars in 1376 bytes} [Thu Jun 23 19:48:23 2022] GET /identity/v3/role_assignments?group.id=f5e4c696c29b4f179b9957e5743c09b1&scope.project.id=4c94dfce039f4ff29bec2f9831ec1748&role.id=860491bfbb574772a7a213d6a25962e0 => generated 611 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:23.414413 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f54a19-5cc8-4d85-9e50-ef615ccaf40c None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:23.414728 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f54a19-5cc8-4d85-9e50-ef615ccaf40c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:23.414975 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f54a19-5cc8-4d85-9e50-ef615ccaf40c None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:23.416415 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 216/429] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:48:23.426118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83720e4-3a39-4e64-ab29-dd1c8f7e29bd None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:23.426458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83720e4-3a39-4e64-ab29-dd1c8f7e29bd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:23.426681 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83720e4-3a39-4e64-ab29-dd1c8f7e29bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:23.450227 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c83720e4-3a39-4e64-ab29-dd1c8f7e29bd None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:23.453427 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c83720e4-3a39-4e64-ab29-dd1c8f7e29bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:23.493165 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c542cba-27ab-4efe-9226-23ddb519c1ec None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:23.513910 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:23.542413 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:23.565921 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940e8fe-9d98-43e0-bbed-74cea654e6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:23.566800 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940e8fe-9d98-43e0-bbed-74cea654e6cb None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:23.567083 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940e8fe-9d98-43e0-bbed-74cea654e6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:23.567359 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940e8fe-9d98-43e0-bbed-74cea654e6cb None admin] PATH_INFO: `/v3/projects/6c9c90447a664a9fa575e9c13ef585f0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:23.569901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940e8fe-9d98-43e0-bbed-74cea654e6cb None admin] RBAC: Authorizing `identity:get_project(project_id=6c9c90447a664a9fa575e9c13ef585f0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:23.571378 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940e8fe-9d98-43e0-bbed-74cea654e6cb None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:23.573105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 214/430] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:48:23 2022] GET /identity/v3/projects/6c9c90447a664a9fa575e9c13ef585f0 => generated 302 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:23.581351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940e8fe-9d98-43e0-bbed-74cea654e6cb None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:23.584940 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 217/431] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 19:48:23.624074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83720e4-3a39-4e64-ab29-dd1c8f7e29bd None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:23.710432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940e8fe-9d98-43e0-bbed-74cea654e6cb None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:23.762576 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d3d01-ff76-45c4-a6a2-40d688430983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:23.762948 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83720e4-3a39-4e64-ab29-dd1c8f7e29bd None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:23.763463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d3d01-ff76-45c4-a6a2-40d688430983 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:23.763798 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d3d01-ff76-45c4-a6a2-40d688430983 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:23.764076 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d3d01-ff76-45c4-a6a2-40d688430983 None admin] PATH_INFO: `/v3/groups/f5e4c696c29b4f179b9957e5743c09b1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:23.765454 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d3d01-ff76-45c4-a6a2-40d688430983 None admin] RBAC: Authorizing `identity:get_group(group_id=f5e4c696c29b4f179b9957e5743c09b1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:23.767133 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d3d01-ff76-45c4-a6a2-40d688430983 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:23.768806 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 215/432] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jun 23 19:48:23 2022] GET /identity/v3/groups/f5e4c696c29b4f179b9957e5743c09b1 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:23.776083 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d3d01-ff76-45c4-a6a2-40d688430983 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:23.818097 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440f606-9c18-436c-b943-7697681f22de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:23.820861 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440f606-9c18-436c-b943-7697681f22de None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:23.821395 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440f606-9c18-436c-b943-7697681f22de None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:23.821883 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440f606-9c18-436c-b943-7697681f22de None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:23.836810 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440f606-9c18-436c-b943-7697681f22de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:23.847559 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:23.848907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-b440f606-9c18-436c-b943-7697681f22de None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 19:48:23.848907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:23.848907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:23.848907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:23.848907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:23.848907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:23.848907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:23.848907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:23.848907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:23.848907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:23.848907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:23.848907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:23.848907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:23.848907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:23.848907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:23.848907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:23.849949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:23.849949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:23.849949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:23.849949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:23.849949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:23.849949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:23.849949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:23.849949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:23.849949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:23.849949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:23.849949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:23.849949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:23.849949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:23.849949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:23.849949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:23.849949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:23.850743 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:23.850743 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:23.850743 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:23.850743 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:23.850743 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:23.850743 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:23.850743 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:23.850743 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:23.850743 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:23.850743 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:23.850743 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 19:48:23.850743 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:23.853423 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 218/433] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jun 23 19:48:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:23.861270 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440f606-9c18-436c-b943-7697681f22de None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:23.901885 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:23.907758 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d3d01-ff76-45c4-a6a2-40d688430983 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:23.956539 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78925f8e-5930-4e84-97ea-6bb67305cb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:23.957204 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78925f8e-5930-4e84-97ea-6bb67305cb5e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:23.957486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78925f8e-5930-4e84-97ea-6bb67305cb5e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:23.957759 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78925f8e-5930-4e84-97ea-6bb67305cb5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:23.958604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78925f8e-5930-4e84-97ea-6bb67305cb5e None admin] RBAC: Adding query filter params (group.id=f5e4c696c29b4f179b9957e5743c09b1, role.id=860491bfbb574772a7a213d6a25962e0, scope.project.id=6c9c90447a664a9fa575e9c13ef585f0) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:23.958946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78925f8e-5930-4e84-97ea-6bb67305cb5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:23.960425 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78925f8e-5930-4e84-97ea-6bb67305cb5e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:23.967443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 216/434] 10.222.0.44 () {56 vars in 1376 bytes} [Thu Jun 23 19:48:23 2022] GET /identity/v3/role_assignments?group.id=f5e4c696c29b4f179b9957e5743c09b1&scope.project.id=6c9c90447a664a9fa575e9c13ef585f0&role.id=860491bfbb574772a7a213d6a25962e0 => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:24.005273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440f606-9c18-436c-b943-7697681f22de None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:24.045295 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe05b10-32e1-4e08-8c03-15baf271bb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:24.046338 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe05b10-32e1-4e08-8c03-15baf271bb45 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:24.046844 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe05b10-32e1-4e08-8c03-15baf271bb45 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:24.047447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe05b10-32e1-4e08-8c03-15baf271bb45 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:24.048500 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe05b10-32e1-4e08-8c03-15baf271bb45 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:24.049019 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe05b10-32e1-4e08-8c03-15baf271bb45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:24.050640 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe05b10-32e1-4e08-8c03-15baf271bb45 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:24.061111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 219/435] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:48:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:24.067292 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78925f8e-5930-4e84-97ea-6bb67305cb5e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:24.187470 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cd034-9867-42d5-8126-5e9593885be3 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:24.188016 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cd034-9867-42d5-8126-5e9593885be3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:24.188284 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cd034-9867-42d5-8126-5e9593885be3 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:24.189716 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 220/436] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:48:24.190254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78925f8e-5930-4e84-97ea-6bb67305cb5e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:24.197886 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436451c-0dcf-45af-85bb-e7ead946bbb1 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:24.198167 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436451c-0dcf-45af-85bb-e7ead946bbb1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:24.198439 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436451c-0dcf-45af-85bb-e7ead946bbb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:24.216697 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:24.223137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8436451c-0dcf-45af-85bb-e7ead946bbb1 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:24.229204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8436451c-0dcf-45af-85bb-e7ead946bbb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:24.234926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880bcd05-7486-44b5-90f8-e54204381f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:24.235581 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bcd05-7486-44b5-90f8-e54204381f53 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:24.235860 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bcd05-7486-44b5-90f8-e54204381f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:24.236147 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bcd05-7486-44b5-90f8-e54204381f53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:24.309558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 221/437] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:24.319293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880bcd05-7486-44b5-90f8-e54204381f53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:24.320818 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880bcd05-7486-44b5-90f8-e54204381f53 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:24.333339 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436451c-0dcf-45af-85bb-e7ead946bbb1 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:24.351781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:24.433136 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 217/438] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:48:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:48:24.434451 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaffb64-689a-48ae-b20e-7a790cbf4e5a None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:24.434738 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaffb64-689a-48ae-b20e-7a790cbf4e5a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:24.435015 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaffb64-689a-48ae-b20e-7a790cbf4e5a None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:24.436356 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 218/439] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:24.438690 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaffb64-689a-48ae-b20e-7a790cbf4e5a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:24.442513 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436451c-0dcf-45af-85bb-e7ead946bbb1 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:24.462403 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:24.476516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e1f3cc-70ba-4f75-8221-bb5f54e015a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:24.477318 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1f3cc-70ba-4f75-8221-bb5f54e015a5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:24.478221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1f3cc-70ba-4f75-8221-bb5f54e015a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:24.478221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1f3cc-70ba-4f75-8221-bb5f54e015a5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:24.488319 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e1f3cc-70ba-4f75-8221-bb5f54e015a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:24.496899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-e0e1f3cc-70ba-4f75-8221-bb5f54e015a5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 19:48:24.496899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:24.496899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:24.496899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:24.496899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:24.496899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:24.496899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:24.496899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:24.496899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:24.496899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:24.496899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:24.496899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:24.496899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:24.496899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:24.496899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:24.496899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:24.497813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:24.497813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:24.497813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:24.497813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:24.497813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:24.497813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:24.497813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:24.497813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:24.497813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:24.497813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:24.497813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:24.497813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:24.497813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:24.497813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:24.497813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:24.497813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:24.498605 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:24.498605 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:24.498605 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:24.498605 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:24.498605 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:24.498605 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:24.498605 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:24.498605 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:24.498605 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:24.498605 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:24.498605 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 19:48:24.498605 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:24.498605 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 222/440] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jun 23 19:48:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:24.499446 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c0e05-7715-450e-afd0-ace860b5d742 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:24.499446 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c0e05-7715-450e-afd0-ace860b5d742 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:24.499738 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c0e05-7715-450e-afd0-ace860b5d742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:24.515930 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-de8c0e05-7715-450e-afd0-ace860b5d742 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:24.517135 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-de8c0e05-7715-450e-afd0-ace860b5d742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:24.541759 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaffb64-689a-48ae-b20e-7a790cbf4e5a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:24.576151 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7df22-389d-44f7-b76b-6c2c867085be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:24.577032 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7df22-389d-44f7-b76b-6c2c867085be None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:24.577313 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7df22-389d-44f7-b76b-6c2c867085be None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:24.577580 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7df22-389d-44f7-b76b-6c2c867085be None admin] PATH_INFO: `/v3/projects/4c94dfce039f4ff29bec2f9831ec1748` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:24.578760 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df7df22-389d-44f7-b76b-6c2c867085be None admin] RBAC: Authorizing `identity:get_project(project_id=4c94dfce039f4ff29bec2f9831ec1748)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:24.580206 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df7df22-389d-44f7-b76b-6c2c867085be None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:24.581909 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 219/441] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:48:24 2022] GET /identity/v3/projects/4c94dfce039f4ff29bec2f9831ec1748 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:24.592399 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 223/442] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:24.592551 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7df22-389d-44f7-b76b-6c2c867085be None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:24.592635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c0e05-7715-450e-afd0-ace860b5d742 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:24.592635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:24.681123 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:24.684715 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c0e05-7715-450e-afd0-ace860b5d742 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:24.690609 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7df22-389d-44f7-b76b-6c2c867085be None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:24.718332 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f8fd2-bdca-4214-82ff-775ab3a27ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:24.719178 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f8fd2-bdca-4214-82ff-775ab3a27ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:24.719454 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f8fd2-bdca-4214-82ff-775ab3a27ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:24.719721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f8fd2-bdca-4214-82ff-775ab3a27ec0 None admin] PATH_INFO: `/v3/groups/f5e4c696c29b4f179b9957e5743c09b1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:24.720900 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f8fd2-bdca-4214-82ff-775ab3a27ec0 None admin] RBAC: Authorizing `identity:get_group(group_id=f5e4c696c29b4f179b9957e5743c09b1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:24.722341 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f8fd2-bdca-4214-82ff-775ab3a27ec0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:24.723931 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 224/443] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jun 23 19:48:24 2022] GET /identity/v3/groups/f5e4c696c29b4f179b9957e5743c09b1 => generated 226 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:24.724129 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead49cb-c19a-4e83-a4e9-eed19193f6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:24.725222 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f8fd2-bdca-4214-82ff-775ab3a27ec0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:24.725392 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead49cb-c19a-4e83-a4e9-eed19193f6c9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:24.725656 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead49cb-c19a-4e83-a4e9-eed19193f6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:24.725970 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead49cb-c19a-4e83-a4e9-eed19193f6c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:24.726659 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cead49cb-c19a-4e83-a4e9-eed19193f6c9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:24.726966 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cead49cb-c19a-4e83-a4e9-eed19193f6c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:24.728354 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cead49cb-c19a-4e83-a4e9-eed19193f6c9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:24.736327 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 220/444] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:48:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:24.741733 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead49cb-c19a-4e83-a4e9-eed19193f6c9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:24.792255 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:24.822272 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f8fd2-bdca-4214-82ff-775ab3a27ec0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:24.837804 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead49cb-c19a-4e83-a4e9-eed19193f6c9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:24.854695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347053b-c0f6-480a-ad34-dab2435f5b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:24.855465 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347053b-c0f6-480a-ad34-dab2435f5b7f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:24.855748 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347053b-c0f6-480a-ad34-dab2435f5b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:24.856024 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347053b-c0f6-480a-ad34-dab2435f5b7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:24.867626 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7347053b-c0f6-480a-ad34-dab2435f5b7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:24.872086 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4367fb-6c1f-4261-bb19-99038169e15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:24.872750 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4367fb-6c1f-4261-bb19-99038169e15b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:24.873037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4367fb-6c1f-4261-bb19-99038169e15b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:24.873314 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4367fb-6c1f-4261-bb19-99038169e15b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:24.874140 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4367fb-6c1f-4261-bb19-99038169e15b None admin] RBAC: Adding query filter params (group.id=f5e4c696c29b4f179b9957e5743c09b1, role.id=03e4cfc753754643ae08fa909647cfe3, scope.project.id=4c94dfce039f4ff29bec2f9831ec1748) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:24.874452 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4367fb-6c1f-4261-bb19-99038169e15b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:24.875111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-7347053b-c0f6-480a-ad34-dab2435f5b7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:48:24.875111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:24.875111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:24.875111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:24.875111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:24.875111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:24.875111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:24.875111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:24.875111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:24.875111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:24.875111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:24.875111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:24.875111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:24.875111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:24.875111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:24.875111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:24.875964 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:24.875964 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:24.875964 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:24.875964 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:24.875964 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:24.875964 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:24.875964 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:24.875964 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:24.875964 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:24.875964 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:24.875964 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:24.875964 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:24.875964 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:24.875964 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:24.875964 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:24.875964 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:24.876792 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:24.876792 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:24.876792 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:24.876792 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:24.876792 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:24.876792 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:24.876792 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:24.876792 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:24.876792 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:24.876792 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:24.876792 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:48:24.876792 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:24.877392 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4367fb-6c1f-4261-bb19-99038169e15b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:24.877457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 225/445] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:48:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:24.877457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347053b-c0f6-480a-ad34-dab2435f5b7f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:24.896357 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 221/446] 10.222.0.44 () {56 vars in 1376 bytes} [Thu Jun 23 19:48:24 2022] GET /identity/v3/role_assignments?group.id=f5e4c696c29b4f179b9957e5743c09b1&scope.project.id=4c94dfce039f4ff29bec2f9831ec1748&role.id=03e4cfc753754643ae08fa909647cfe3 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:24.896357 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4367fb-6c1f-4261-bb19-99038169e15b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:24.906377 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:24.990006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347053b-c0f6-480a-ad34-dab2435f5b7f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:24.999986 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4367fb-6c1f-4261-bb19-99038169e15b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:25.024090 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:25.030396 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76eb07f-284c-4863-b9c4-df70a6fd8aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:25.031210 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76eb07f-284c-4863-b9c4-df70a6fd8aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:25.031495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76eb07f-284c-4863-b9c4-df70a6fd8aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:25.031767 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76eb07f-284c-4863-b9c4-df70a6fd8aa2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:25.036959 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4770ab4-fbd4-44bc-8ac6-c428e696f442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:25.038190 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4770ab4-fbd4-44bc-8ac6-c428e696f442 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:25.038479 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4770ab4-fbd4-44bc-8ac6-c428e696f442 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:25.038759 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4770ab4-fbd4-44bc-8ac6-c428e696f442 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:25.040409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4770ab4-fbd4-44bc-8ac6-c428e696f442 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:25.040409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4770ab4-fbd4-44bc-8ac6-c428e696f442 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:25.041228 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4770ab4-fbd4-44bc-8ac6-c428e696f442 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:25.050822 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 222/447] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:25.057355 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4770ab4-fbd4-44bc-8ac6-c428e696f442 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:25.109187 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76eb07f-284c-4863-b9c4-df70a6fd8aa2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:25.110997 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76eb07f-284c-4863-b9c4-df70a6fd8aa2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:25.151918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:25.182180 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4770ab4-fbd4-44bc-8ac6-c428e696f442 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:25.221862 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204f49d-66dd-467f-9cf8-d1c94c7aef50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:25.222518 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204f49d-66dd-467f-9cf8-d1c94c7aef50 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:25.222797 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204f49d-66dd-467f-9cf8-d1c94c7aef50 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:25.223067 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4204f49d-66dd-467f-9cf8-d1c94c7aef50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:25.233858 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 226/448] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:48:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:48:25.239956 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76eb07f-284c-4863-b9c4-df70a6fd8aa2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:25.279424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:25.300920 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4204f49d-66dd-467f-9cf8-d1c94c7aef50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:25.302418 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4204f49d-66dd-467f-9cf8-d1c94c7aef50 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:25.374316 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76eb07f-284c-4863-b9c4-df70a6fd8aa2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:25.396076 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:25.411657 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4eeda9-65c2-4f09-aea7-dfc44d01275d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:25.413128 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4eeda9-65c2-4f09-aea7-dfc44d01275d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:25.413404 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4eeda9-65c2-4f09-aea7-dfc44d01275d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:25.413671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4eeda9-65c2-4f09-aea7-dfc44d01275d None admin] PATH_INFO: `/v3/projects/6c9c90447a664a9fa575e9c13ef585f0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:25.414910 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4eeda9-65c2-4f09-aea7-dfc44d01275d None admin] RBAC: Authorizing `identity:get_project(project_id=6c9c90447a664a9fa575e9c13ef585f0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:25.416380 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4eeda9-65c2-4f09-aea7-dfc44d01275d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:25.422967 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 227/449] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:48:25 2022] GET /identity/v3/projects/6c9c90447a664a9fa575e9c13ef585f0 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:25.423084 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 223/450] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:48:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 19:48:25.423716 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4eeda9-65c2-4f09-aea7-dfc44d01275d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:25.423824 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204f49d-66dd-467f-9cf8-d1c94c7aef50 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:25.517087 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:25.519825 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4204f49d-66dd-467f-9cf8-d1c94c7aef50 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:25.519976 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4eeda9-65c2-4f09-aea7-dfc44d01275d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:25.553093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca04e7-d649-4aa4-b4ae-67752733998f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:25.553269 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ba93d-8dc5-4b81-bc85-576deb424732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:25.553878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca04e7-d649-4aa4-b4ae-67752733998f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:25.554114 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ba93d-8dc5-4b81-bc85-576deb424732 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:25.554200 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca04e7-d649-4aa4-b4ae-67752733998f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:25.554408 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca04e7-d649-4aa4-b4ae-67752733998f None admin] PATH_INFO: `/v3/projects/e3c8189cd37644e9a49e850123b33202` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:25.554508 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ba93d-8dc5-4b81-bc85-576deb424732 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:25.554731 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ba93d-8dc5-4b81-bc85-576deb424732 None admin] PATH_INFO: `/v3/groups/f5e4c696c29b4f179b9957e5743c09b1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:25.555579 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ca04e7-d649-4aa4-b4ae-67752733998f None admin] RBAC: Authorizing `identity:get_project(project_id=e3c8189cd37644e9a49e850123b33202)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:25.556229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ba93d-8dc5-4b81-bc85-576deb424732 None admin] RBAC: Authorizing `identity:get_group(group_id=f5e4c696c29b4f179b9957e5743c09b1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:25.557058 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ca04e7-d649-4aa4-b4ae-67752733998f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:25.557680 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ba93d-8dc5-4b81-bc85-576deb424732 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:25.558724 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 224/451] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:48:25 2022] GET /identity/v3/projects/e3c8189cd37644e9a49e850123b33202 => generated 344 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:25.559274 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 228/452] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jun 23 19:48:25 2022] GET /identity/v3/groups/f5e4c696c29b4f179b9957e5743c09b1 => generated 226 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:25.564171 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca04e7-d649-4aa4-b4ae-67752733998f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:25.565136 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ba93d-8dc5-4b81-bc85-576deb424732 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:25.610895 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:25.666636 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ba93d-8dc5-4b81-bc85-576deb424732 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:25.671159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca04e7-d649-4aa4-b4ae-67752733998f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:25.700478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64292b2c-19f6-47e0-8b26-339ac66e396b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:25.701117 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64292b2c-19f6-47e0-8b26-339ac66e396b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:25.701397 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64292b2c-19f6-47e0-8b26-339ac66e396b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:25.701676 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64292b2c-19f6-47e0-8b26-339ac66e396b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:25.702495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64292b2c-19f6-47e0-8b26-339ac66e396b None admin] RBAC: Adding query filter params (group.id=f5e4c696c29b4f179b9957e5743c09b1, role.id=03e4cfc753754643ae08fa909647cfe3, scope.project.id=6c9c90447a664a9fa575e9c13ef585f0) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:25.702796 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64292b2c-19f6-47e0-8b26-339ac66e396b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:25.704215 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64292b2c-19f6-47e0-8b26-339ac66e396b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:25.704932 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f80b7-272b-4068-948a-5be7b281528e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:25.705794 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f80b7-272b-4068-948a-5be7b281528e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:25.706240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f80b7-272b-4068-948a-5be7b281528e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:25.706518 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f80b7-272b-4068-948a-5be7b281528e None admin] PATH_INFO: `/v3/groups/f23cb458b0f2437d8574c068528dec1f` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:25.708170 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f80b7-272b-4068-948a-5be7b281528e None admin] RBAC: Authorizing `identity:get_group(group_id=f23cb458b0f2437d8574c068528dec1f)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:25.710229 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f80b7-272b-4068-948a-5be7b281528e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:25.710591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 229/453] 10.222.0.44 () {56 vars in 1376 bytes} [Thu Jun 23 19:48:25 2022] GET /identity/v3/role_assignments?group.id=f5e4c696c29b4f179b9957e5743c09b1&scope.project.id=6c9c90447a664a9fa575e9c13ef585f0&role.id=03e4cfc753754643ae08fa909647cfe3 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:25.715648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 225/454] 10.222.0.44 () {56 vars in 1180 bytes} [Thu Jun 23 19:48:25 2022] GET /identity/v3/groups/f23cb458b0f2437d8574c068528dec1f => generated 229 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:25.717887 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64292b2c-19f6-47e0-8b26-339ac66e396b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:25.751653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:25.825967 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64292b2c-19f6-47e0-8b26-339ac66e396b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:25.861741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ca117-14a1-4910-a37c-fd8191ec0955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:25.862388 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ca117-14a1-4910-a37c-fd8191ec0955 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:25.862671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ca117-14a1-4910-a37c-fd8191ec0955 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:25.862945 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ca117-14a1-4910-a37c-fd8191ec0955 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:25.863771 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ca117-14a1-4910-a37c-fd8191ec0955 None admin] RBAC: Adding query filter params (group.id=f23cb458b0f2437d8574c068528dec1f, role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb, scope.project.id=e3c8189cd37644e9a49e850123b33202) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:25.864088 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ca117-14a1-4910-a37c-fd8191ec0955 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:25.866117 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ca117-14a1-4910-a37c-fd8191ec0955 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:25.871932 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 230/455] 10.222.0.44 () {56 vars in 1376 bytes} [Thu Jun 23 19:48:25 2022] GET /identity/v3/role_assignments?group.id=f23cb458b0f2437d8574c068528dec1f&scope.project.id=e3c8189cd37644e9a49e850123b33202&role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:28.294564 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4ffba-95b6-4392-b9dd-d15999ec2c6a None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:28.295551 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4ffba-95b6-4392-b9dd-d15999ec2c6a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:28.296018 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4ffba-95b6-4392-b9dd-d15999ec2c6a None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:28.297904 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 226/456] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:28.303484 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee66ff-0713-40cf-8778-d1a8370db3ff None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:28.303911 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee66ff-0713-40cf-8778-d1a8370db3ff None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:28.304026 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee66ff-0713-40cf-8778-d1a8370db3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:28.329217 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5dee66ff-0713-40cf-8778-d1a8370db3ff None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:28.330655 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5dee66ff-0713-40cf-8778-d1a8370db3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:28.426603 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 231/457] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:28.440721 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e6939-ce87-4163-a387-653e241f5401 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:28.441095 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e6939-ce87-4163-a387-653e241f5401 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:28.441436 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e6939-ce87-4163-a387-653e241f5401 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:28.442849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 227/458] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:28.448719 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ebae0-6906-403b-9f12-51497a168d5a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:28.449221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ebae0-6906-403b-9f12-51497a168d5a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:28.449691 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ebae0-6906-403b-9f12-51497a168d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:28.465550 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3e6939-ce87-4163-a387-653e241f5401 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:28.469325 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f29ebae0-6906-403b-9f12-51497a168d5a None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:28.472323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f29ebae0-6906-403b-9f12-51497a168d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:28.493665 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:28.578917 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 232/459] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:28.617692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6d9e0-d25d-4131-a0f9-f9dcb6eac0be None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:28.617993 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6d9e0-d25d-4131-a0f9-f9dcb6eac0be None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:28.618251 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6d9e0-d25d-4131-a0f9-f9dcb6eac0be None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:28.619870 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 233/460] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:28.624839 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9dae6a-a8c5-439a-9e94-9ca7ba588d6f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:28.625135 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9dae6a-a8c5-439a-9e94-9ca7ba588d6f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:28.625417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9dae6a-a8c5-439a-9e94-9ca7ba588d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:28.625936 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3e6939-ce87-4163-a387-653e241f5401 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:28.642184 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:28.649362 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-cb9dae6a-a8c5-439a-9e94-9ca7ba588d6f None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:28.650853 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-cb9dae6a-a8c5-439a-9e94-9ca7ba588d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:28.664425 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc07fa-a9c6-4e81-87e2-5a8abddcc388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:28.665220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc07fa-a9c6-4e81-87e2-5a8abddcc388 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:28.665499 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc07fa-a9c6-4e81-87e2-5a8abddcc388 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:28.665767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc07fa-a9c6-4e81-87e2-5a8abddcc388 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:28.676901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc07fa-a9c6-4e81-87e2-5a8abddcc388 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:28.678828 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc07fa-a9c6-4e81-87e2-5a8abddcc388 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:28.698558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-38bc07fa-a9c6-4e81-87e2-5a8abddcc388 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:28.698558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:28.698558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:28.698558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:28.698558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:28.698558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:28.698558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:28.698558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:28.698558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:28.698558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:28.698558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:28.698558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:28.698558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:28.698558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:28.698558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:28.699661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:28.699661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:28.699661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:28.699661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:28.699661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:28.699661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:28.699661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:28.699661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:28.699661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:28.699661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:28.699661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:28.699661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:28.699661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:28.699661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:28.699661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:28.699661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:28.699661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:28.700629 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:28.700629 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:28.700629 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:28.700629 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:28.702659 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 228/461] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:28.703744 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc07fa-a9c6-4e81-87e2-5a8abddcc388 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:28.732645 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 234/462] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 19:48:28.733431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9dae6a-a8c5-439a-9e94-9ca7ba588d6f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:28.806329 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc07fa-a9c6-4e81-87e2-5a8abddcc388 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:28.841239 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d4c46-bf57-4876-8229-3ef95b1f13d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:28.842238 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d4c46-bf57-4876-8229-3ef95b1f13d2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:28.842663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d4c46-bf57-4876-8229-3ef95b1f13d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:28.843098 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3d4c46-bf57-4876-8229-3ef95b1f13d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:28.848661 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:28.854032 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d4c46-bf57-4876-8229-3ef95b1f13d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:28.855699 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3d4c46-bf57-4876-8229-3ef95b1f13d2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:28.857727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9dae6a-a8c5-439a-9e94-9ca7ba588d6f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:28.874863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-ef3d4c46-bf57-4876-8229-3ef95b1f13d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:28.874863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:28.874863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:28.874863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:28.874863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:28.874863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:28.874863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:28.874863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:28.874863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:28.874863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:28.874863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:28.874863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:28.874863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:28.874863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:28.874863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:28.876993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:28.876993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:28.876993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:28.876993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:28.876993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:28.876993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:28.876993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:28.876993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:28.876993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:28.876993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:28.876993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:28.876993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:28.876993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:28.876993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:28.876993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:28.876993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:28.876993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:28.877855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:28.877855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:28.877855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:28.877855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:28.879081 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 229/463] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:28.880109 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d4c46-bf57-4876-8229-3ef95b1f13d2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:28.899283 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e4d5e-9099-4513-a57d-e00201fa4be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:28.900014 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e4d5e-9099-4513-a57d-e00201fa4be4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:28.900322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e4d5e-9099-4513-a57d-e00201fa4be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:28.900617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e4d5e-9099-4513-a57d-e00201fa4be4 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:28.901458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e4d5e-9099-4513-a57d-e00201fa4be4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:28.901770 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e4d5e-9099-4513-a57d-e00201fa4be4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:28.902454 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:28.904076 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e4d5e-9099-4513-a57d-e00201fa4be4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:28.916089 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 235/464] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:28.917044 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e4d5e-9099-4513-a57d-e00201fa4be4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:28.984883 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3d4c46-bf57-4876-8229-3ef95b1f13d2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:29.026678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1341c0-87b3-4925-b34e-dbe2808e5d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:29.027642 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1341c0-87b3-4925-b34e-dbe2808e5d15 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:29.027927 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1341c0-87b3-4925-b34e-dbe2808e5d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:29.028206 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1341c0-87b3-4925-b34e-dbe2808e5d15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:29.029928 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e4d5e-9099-4513-a57d-e00201fa4be4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:29.043081 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1341c0-87b3-4925-b34e-dbe2808e5d15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:29.046309 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1341c0-87b3-4925-b34e-dbe2808e5d15 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:29.067739 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-5e1341c0-87b3-4925-b34e-dbe2808e5d15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:29.067739 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:29.067739 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:29.067739 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:29.067739 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:29.067739 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:29.067739 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:29.067739 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:29.067739 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:29.067739 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:29.067739 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:29.067739 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:29.067739 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:29.067739 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:29.067739 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:29.068712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:29.068712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:29.068712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:29.068712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:29.068712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:29.068712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:29.068712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:29.068712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:29.068712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:29.068712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:29.068712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:29.068712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:29.068712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:29.068712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:29.068712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:29.068712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:29.068712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:29.070146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:29.070146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:29.070146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:29.070146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:29.070146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 230/465] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:29.070423 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1341c0-87b3-4925-b34e-dbe2808e5d15 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:29.076236 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba86e87-45a4-4dfc-ba0c-582c54c226f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:29.077195 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba86e87-45a4-4dfc-ba0c-582c54c226f1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:29.077641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba86e87-45a4-4dfc-ba0c-582c54c226f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:29.081819 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba86e87-45a4-4dfc-ba0c-582c54c226f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:29.081819 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba86e87-45a4-4dfc-ba0c-582c54c226f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:29.081819 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba86e87-45a4-4dfc-ba0c-582c54c226f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:29.081819 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba86e87-45a4-4dfc-ba0c-582c54c226f1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:29.098686 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 236/466] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:29.100742 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba86e87-45a4-4dfc-ba0c-582c54c226f1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:29.111778 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:29.144508 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:29.174179 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1341c0-87b3-4925-b34e-dbe2808e5d15 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:29.210664 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba86e87-45a4-4dfc-ba0c-582c54c226f1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:29.214479 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546245a7-c338-4c28-a55c-7dc9b06d622d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:29.215236 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546245a7-c338-4c28-a55c-7dc9b06d622d None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:29.215519 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546245a7-c338-4c28-a55c-7dc9b06d622d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:29.215795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546245a7-c338-4c28-a55c-7dc9b06d622d None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:29.216538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546245a7-c338-4c28-a55c-7dc9b06d622d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:29.218536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546245a7-c338-4c28-a55c-7dc9b06d622d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:29.246731 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab02308-9461-4cea-bdd0-9e26fcb925bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:29.247387 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab02308-9461-4cea-bdd0-9e26fcb925bf None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:29.247666 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab02308-9461-4cea-bdd0-9e26fcb925bf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:29.247937 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab02308-9461-4cea-bdd0-9e26fcb925bf None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:29.727517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab02308-9461-4cea-bdd0-9e26fcb925bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:29.727517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab02308-9461-4cea-bdd0-9e26fcb925bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:29.727517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab02308-9461-4cea-bdd0-9e26fcb925bf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:29.727517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 237/467] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:29.727517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab02308-9461-4cea-bdd0-9e26fcb925bf None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:29.727517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:29.728465 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab02308-9461-4cea-bdd0-9e26fcb925bf None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:29.728465 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e80c4-5f65-4a16-b90d-6f6879880dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:29.728465 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e80c4-5f65-4a16-b90d-6f6879880dd5 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:29.728465 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e80c4-5f65-4a16-b90d-6f6879880dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:29.728754 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e80c4-5f65-4a16-b90d-6f6879880dd5 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:29.728754 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e80c4-5f65-4a16-b90d-6f6879880dd5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:29.728754 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e80c4-5f65-4a16-b90d-6f6879880dd5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:29.883180 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 231/468] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jun 23 19:48:29 2022] POST /identity/v3/users => generated 246 bytes in 813 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:48:29.884527 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b8cea-678d-4f83-b0c7-942ef16a9db3 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:29.884813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b8cea-678d-4f83-b0c7-942ef16a9db3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:29.885089 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b8cea-678d-4f83-b0c7-942ef16a9db3 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:29.888433 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 232/469] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:29.888433 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b8cea-678d-4f83-b0c7-942ef16a9db3 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:29.902789 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 238/470] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jun 23 19:48:29 2022] POST /identity/v3/users => generated 248 bytes in 641 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:48:29.904500 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169adfe-37f7-43c5-be83-df9e30af567f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:29.905067 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169adfe-37f7-43c5-be83-df9e30af567f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:29.905545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169adfe-37f7-43c5-be83-df9e30af567f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:29.938521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-3169adfe-37f7-43c5-be83-df9e30af567f None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:29.942363 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3169adfe-37f7-43c5-be83-df9e30af567f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:30.080347 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 239/471] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:30.080347 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff27102-d73c-4693-b7a5-82476566cb34 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:30.080347 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff27102-d73c-4693-b7a5-82476566cb34 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:30.080347 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff27102-d73c-4693-b7a5-82476566cb34 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:30.081235 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 240/472] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:30.086361 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86334c69-1ef7-49cf-9928-f47fd3b46f6c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:30.086648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86334c69-1ef7-49cf-9928-f47fd3b46f6c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:30.086921 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86334c69-1ef7-49cf-9928-f47fd3b46f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:30.088497 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b8cea-678d-4f83-b0c7-942ef16a9db3 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:30.110483 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-86334c69-1ef7-49cf-9928-f47fd3b46f6c None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:30.111891 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-86334c69-1ef7-49cf-9928-f47fd3b46f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:30.119719 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:30.129714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56920983-a0df-47fa-b51c-6251aae7d212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:30.130378 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56920983-a0df-47fa-b51c-6251aae7d212 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:30.131188 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56920983-a0df-47fa-b51c-6251aae7d212 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:30.131537 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56920983-a0df-47fa-b51c-6251aae7d212 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:30.137152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56920983-a0df-47fa-b51c-6251aae7d212 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:30.137152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56920983-a0df-47fa-b51c-6251aae7d212 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:30.170326 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:30.190486 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 241/473] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 19:48:30.191782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86334c69-1ef7-49cf-9928-f47fd3b46f6c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:30.294566 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86334c69-1ef7-49cf-9928-f47fd3b46f6c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:30.328885 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729a9ff-c3ed-47fa-a6a4-beca883a4716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:30.329673 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729a9ff-c3ed-47fa-a6a4-beca883a4716 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:30.329956 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729a9ff-c3ed-47fa-a6a4-beca883a4716 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:30.330229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729a9ff-c3ed-47fa-a6a4-beca883a4716 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:30.340743 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b729a9ff-c3ed-47fa-a6a4-beca883a4716 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:30.342144 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b729a9ff-c3ed-47fa-a6a4-beca883a4716 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:30.359354 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-b729a9ff-c3ed-47fa-a6a4-beca883a4716 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:30.359354 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:30.359354 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:30.359354 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:30.359354 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:30.359354 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:30.359354 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:30.359354 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:30.359354 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:30.359354 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:30.359354 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:30.359354 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:30.359354 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:30.359354 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:30.359354 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:30.360261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:30.360261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:30.360261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:30.360261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:30.360261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:30.360261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:30.360261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:30.360261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:30.360261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:30.360261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:30.360261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:30.360261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:30.360261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:30.360261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:30.360261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:30.360261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:30.360261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:30.361694 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:30.361694 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:30.361694 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:30.361694 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:30.361694 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 242/474] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:30.361694 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729a9ff-c3ed-47fa-a6a4-beca883a4716 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:30.458512 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729a9ff-c3ed-47fa-a6a4-beca883a4716 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:30.492730 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78104bb7-1fce-4b3f-85b2-a5b8efd140ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:30.493395 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78104bb7-1fce-4b3f-85b2-a5b8efd140ec None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:30.493679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78104bb7-1fce-4b3f-85b2-a5b8efd140ec None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:30.493954 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78104bb7-1fce-4b3f-85b2-a5b8efd140ec None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:30.494696 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78104bb7-1fce-4b3f-85b2-a5b8efd140ec None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:30.496068 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78104bb7-1fce-4b3f-85b2-a5b8efd140ec None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:31.284853 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.sql.core [None req-78104bb7-1fce-4b3f-85b2-a5b8efd140ec None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 19:48:31.284853 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 23 19:48:31.284853 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [parameters: {'id': '1735cc0743b54fcf9a911d69d3171170', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 23 19:48:31.284853 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80363) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 19:48:31.296392 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 233/475] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jun 23 19:48:29 2022] POST /identity/v3/users => generated 248 bytes in 1405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:48:31.296392 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56920983-a0df-47fa-b51c-6251aae7d212 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:31.302430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-78104bb7-1fce-4b3f-85b2-a5b8efd140ec None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 23 19:48:31.302430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:31.302430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 19:48:31.302430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 19:48:31.302430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 19:48:31.302430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 19:48:31.302430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 19:48:31.302430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 19:48:31.302430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 19:48:31.302430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application conn.query(q) Jun 23 19:48:31.302430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 19:48:31.302430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 19:48:31.302430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 19:48:31.302430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result.read() Jun 23 19:48:31.302430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 19:48:31.305331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 19:48:31.305331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 19:48:31.305331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 19:48:31.305331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 19:48:31.305331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 19:48:31.305331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 19:48:31.305331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 19:48:31.305331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 19:48:31.305331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:31.305331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 19:48:31.305331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:31.305331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:31.305331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 19:48:31.305331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 19:48:31.305331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 23 19:48:31.305331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return ref.to_dict() Jun 23 19:48:31.305331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 19:48:31.305331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application next(self.gen) Jun 23 19:48:31.306221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 19:48:31.306221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application yield resource Jun 23 19:48:31.306221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 19:48:31.306221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application next(self.gen) Jun 23 19:48:31.306221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 19:48:31.306221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 19:48:31.306221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 19:48:31.306221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 19:48:31.306221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 19:48:31.306221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise self.value Jun 23 19:48:31.306221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 19:48:31.306221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 19:48:31.306221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 19:48:31.306221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application session.commit() Jun 23 19:48:31.306221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 19:48:31.306221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 19:48:31.306221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 19:48:31.307070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 19:48:31.307070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 19:48:31.307070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.session.flush() Jun 23 19:48:31.307070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 19:48:31.307070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 19:48:31.307070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 19:48:31.307070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 19:48:31.307070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 19:48:31.307070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application compat.raise_( Jun 23 19:48:31.307070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 19:48:31.307070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception Jun 23 19:48:31.307070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 19:48:31.307070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 19:48:31.307070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 19:48:31.307070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 19:48:31.307070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 19:48:31.307070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 19:48:31.307899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 19:48:31.307899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 19:48:31.307899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 23 19:48:31.307899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 23 19:48:31.307899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 19:48:31.307899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 19:48:31.307899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 19:48:31.307899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 19:48:31.307899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 19:48:31.307899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 19:48:31.307899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 19:48:31.307899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 19:48:31.307899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 19:48:31.307899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 19:48:31.307899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 19:48:31.307899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception Jun 23 19:48:31.308809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 19:48:31.308809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 19:48:31.308809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 19:48:31.308809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 19:48:31.308809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 19:48:31.308809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 19:48:31.308809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 19:48:31.308809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application conn.query(q) Jun 23 19:48:31.308809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 19:48:31.308809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 19:48:31.308809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 19:48:31.308809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result.read() Jun 23 19:48:31.308809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 19:48:31.308809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 19:48:31.308809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 19:48:31.308809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 19:48:31.309594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 19:48:31.309594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 19:48:31.309594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 19:48:31.309594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 19:48:31.309594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 19:48:31.309594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 23 19:48:31.309594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [parameters: {'id': '1735cc0743b54fcf9a911d69d3171170', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 23 19:48:31.309594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 19:48:31.309594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:31.309594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:48:31.309594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:31.309594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:31.309594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:31.309594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:31.309594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:31.309594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:31.313032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:31.313032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:31.313032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:31.313032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:31.313032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:31.313032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:31.313032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 23 19:48:31.313032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 23 19:48:31.313032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:31.313032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:31.313032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 23 19:48:31.313032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 23 19:48:31.313032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 19:48:31.313032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 19:48:31.313032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 23 19:48:31.313032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:31.315006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 243/476] 10.222.0.44 () {60 vars in 1131 bytes} [Thu Jun 23 19:48:30 2022] POST /identity/v3/roles => generated 144 bytes in 943 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:31.316915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78104bb7-1fce-4b3f-85b2-a5b8efd140ec None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:31.434326 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78104bb7-1fce-4b3f-85b2-a5b8efd140ec None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:31.463327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:31.477546 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc497a-fe56-4f55-9773-db7f1fbb7283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:31.478656 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc497a-fe56-4f55-9773-db7f1fbb7283 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:31.479507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc497a-fe56-4f55-9773-db7f1fbb7283 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:31.479985 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc497a-fe56-4f55-9773-db7f1fbb7283 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:31.491203 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:31.497414 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc497a-fe56-4f55-9773-db7f1fbb7283 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:31.505045 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56920983-a0df-47fa-b51c-6251aae7d212 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:31.508953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-25dc497a-fe56-4f55-9773-db7f1fbb7283 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 19:48:31.508953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:31.508953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:31.508953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:31.508953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:31.508953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:31.508953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:31.508953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:31.508953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:31.508953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:31.508953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:31.508953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:31.508953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:31.508953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:31.508953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:31.508953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:31.517718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:31.517718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:31.517718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:31.517718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:31.517718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:31.517718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:31.517718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:31.517718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:31.517718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:31.517718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:31.517718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:31.517718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:31.517718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:31.517718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:31.517718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:31.517718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:31.518490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:31.518490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:31.518490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:31.518490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:31.518490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:31.518490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:31.518490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:31.518490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:31.518490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:31.518490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:31.518490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 19:48:31.518490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:31.518490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 244/477] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jun 23 19:48:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:31.518490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc497a-fe56-4f55-9773-db7f1fbb7283 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:31.544602 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc4a8ff-ad4c-4983-a84f-fa6dbe3269ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:31.545397 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4a8ff-ad4c-4983-a84f-fa6dbe3269ef None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:31.545690 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4a8ff-ad4c-4983-a84f-fa6dbe3269ef None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:31.545970 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc4a8ff-ad4c-4983-a84f-fa6dbe3269ef None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:31.546705 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc4a8ff-ad4c-4983-a84f-fa6dbe3269ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:31.547017 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc4a8ff-ad4c-4983-a84f-fa6dbe3269ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:31.548502 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc4a8ff-ad4c-4983-a84f-fa6dbe3269ef None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:31.573113 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 234/478] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:31.578637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc4a8ff-ad4c-4983-a84f-fa6dbe3269ef None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:31.635456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:31.640853 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc497a-fe56-4f55-9773-db7f1fbb7283 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:31.677448 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7020c-ab85-432d-85c6-5b3a355a8820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:31.678391 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7020c-ab85-432d-85c6-5b3a355a8820 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:31.678687 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7020c-ab85-432d-85c6-5b3a355a8820 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:31.678968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7020c-ab85-432d-85c6-5b3a355a8820 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:31.679730 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c7020c-ab85-432d-85c6-5b3a355a8820 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:31.680042 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c7020c-ab85-432d-85c6-5b3a355a8820 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:31.681923 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c7020c-ab85-432d-85c6-5b3a355a8820 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:31.696777 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 245/479] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:48:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:31.699222 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc4a8ff-ad4c-4983-a84f-fa6dbe3269ef None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:31.740488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dceb42-1f94-469d-b817-31ef10a2bdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:31.741634 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dceb42-1f94-469d-b817-31ef10a2bdd5 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:31.742189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dceb42-1f94-469d-b817-31ef10a2bdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:31.742663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dceb42-1f94-469d-b817-31ef10a2bdd5 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:31.743679 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dceb42-1f94-469d-b817-31ef10a2bdd5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:31.745466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dceb42-1f94-469d-b817-31ef10a2bdd5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:31.850927 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 235/480] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jun 23 19:48:31 2022] POST /identity/v3/users => generated 249 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:48:32.141716 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9152c98-828b-4cce-8c51-fe652a6c503b None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:32.142087 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9152c98-828b-4cce-8c51-fe652a6c503b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:32.142292 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9152c98-828b-4cce-8c51-fe652a6c503b None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:32.143801 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 246/481] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:32.149477 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08908649-4d86-4477-8ede-736d5015879a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:32.149760 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08908649-4d86-4477-8ede-736d5015879a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:32.150035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08908649-4d86-4477-8ede-736d5015879a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:32.165698 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede77da4-f2bd-4b74-94d3-ca6dd663c63e None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:32.166288 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede77da4-f2bd-4b74-94d3-ca6dd663c63e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:32.166820 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede77da4-f2bd-4b74-94d3-ca6dd663c63e None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:32.168403 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 247/482] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:32.171390 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-08908649-4d86-4477-8ede-736d5015879a None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:32.172681 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-08908649-4d86-4477-8ede-736d5015879a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:32.174098 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e94d9-5052-4c4f-b455-bf40f12494c3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:32.174594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e94d9-5052-4c4f-b455-bf40f12494c3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:32.175068 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e94d9-5052-4c4f-b455-bf40f12494c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:32.198589 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5d7e94d9-5052-4c4f-b455-bf40f12494c3 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:32.199771 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5d7e94d9-5052-4c4f-b455-bf40f12494c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:32.219305 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:32.258872 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 236/483] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 19:48:32.282699 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08908649-4d86-4477-8ede-736d5015879a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:32.288724 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 248/484] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 19:48:32.314560 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e94d9-5052-4c4f-b455-bf40f12494c3 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:32.333775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:32.460095 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08908649-4d86-4477-8ede-736d5015879a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:32.483235 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e94d9-5052-4c4f-b455-bf40f12494c3 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:32.522261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5383b5c-0e86-465b-a702-705c7ce1b2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:32.522489 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624acca1-95b7-4538-9736-7ab9b39484e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:32.523059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5383b5c-0e86-465b-a702-705c7ce1b2b0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:32.523164 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624acca1-95b7-4538-9736-7ab9b39484e9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:32.523335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5383b5c-0e86-465b-a702-705c7ce1b2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:32.523427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624acca1-95b7-4538-9736-7ab9b39484e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:32.523641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5383b5c-0e86-465b-a702-705c7ce1b2b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:32.523739 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624acca1-95b7-4538-9736-7ab9b39484e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:32.544180 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5383b5c-0e86-465b-a702-705c7ce1b2b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:32.548846 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624acca1-95b7-4538-9736-7ab9b39484e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:32.559879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-e5383b5c-0e86-465b-a702-705c7ce1b2b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:32.559879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:32.559879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:32.559879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:32.559879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:32.559879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:32.559879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:32.559879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:32.559879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:32.559879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:32.559879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:32.559879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:32.559879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:32.559879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:32.559879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:32.559879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:32.560852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:32.560852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:32.560852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:32.560852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:32.560852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:32.560852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:32.560852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:32.560852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:32.560852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:32.560852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:32.560852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:32.560852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:32.560852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:32.560852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:32.560852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:32.560852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:32.561647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:32.561647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:32.561647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:32.561647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:32.561647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:32.561647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:32.561647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:32.561647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:32.561647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:32.561647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:32.561647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:32.561647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:32.561647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 249/485] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jun 23 19:48:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:32.562300 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-624acca1-95b7-4538-9736-7ab9b39484e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:32.562300 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:32.562300 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:32.562300 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:32.562300 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:32.562300 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:32.562300 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:32.562300 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:32.562300 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:32.562300 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:32.562300 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:32.562300 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:32.562300 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:32.562300 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:32.562300 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:32.562300 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:32.563379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:32.563379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:32.563379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:32.563379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:32.563379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:32.563379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:32.563379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:32.563379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:32.563379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:32.563379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:32.563379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:32.563379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:32.563379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:32.563379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:32.563379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:32.563379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:32.564147 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:32.564147 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:32.564147 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:32.564147 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:32.564147 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:32.564147 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:32.564147 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:32.564147 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:32.564147 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:32.564147 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:32.564147 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:32.564147 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:32.564147 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 237/486] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jun 23 19:48:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:32.576248 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5383b5c-0e86-465b-a702-705c7ce1b2b0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:32.577522 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624acca1-95b7-4538-9736-7ab9b39484e9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:32.600105 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:32.676997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:32.690057 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624acca1-95b7-4538-9736-7ab9b39484e9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:32.730009 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b6f5c-53a6-4ed2-86c5-76217d7cf029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:32.730593 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b6f5c-53a6-4ed2-86c5-76217d7cf029 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:32.730873 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b6f5c-53a6-4ed2-86c5-76217d7cf029 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:32.731143 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b6f5c-53a6-4ed2-86c5-76217d7cf029 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:32.731892 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424b6f5c-53a6-4ed2-86c5-76217d7cf029 None admin] RBAC: Adding query filter params (name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:32.732196 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424b6f5c-53a6-4ed2-86c5-76217d7cf029 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:32.733619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424b6f5c-53a6-4ed2-86c5-76217d7cf029 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:32.746695 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 238/487] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jun 23 19:48:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:32.752693 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b6f5c-53a6-4ed2-86c5-76217d7cf029 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:32.770617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5383b5c-0e86-465b-a702-705c7ce1b2b0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:32.811638 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7b4c8-2168-45db-a546-c76a20ee95e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:32.812428 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7b4c8-2168-45db-a546-c76a20ee95e4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:32.812980 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7b4c8-2168-45db-a546-c76a20ee95e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:32.813372 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7b4c8-2168-45db-a546-c76a20ee95e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:32.814422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c7b4c8-2168-45db-a546-c76a20ee95e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:32.814738 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c7b4c8-2168-45db-a546-c76a20ee95e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:32.816163 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c7b4c8-2168-45db-a546-c76a20ee95e4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:32.825922 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 250/488] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jun 23 19:48:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:32.831132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7b4c8-2168-45db-a546-c76a20ee95e4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:32.857337 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b6f5c-53a6-4ed2-86c5-76217d7cf029 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:32.892045 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25303eff-0ad8-4f29-886d-ff035294b910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:32.892892 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25303eff-0ad8-4f29-886d-ff035294b910 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:32.893177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25303eff-0ad8-4f29-886d-ff035294b910 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:32.896324 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25303eff-0ad8-4f29-886d-ff035294b910 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:32.912205 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25303eff-0ad8-4f29-886d-ff035294b910 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:32.913702 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25303eff-0ad8-4f29-886d-ff035294b910 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:32.946840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-25303eff-0ad8-4f29-886d-ff035294b910 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:32.946840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:32.946840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:32.946840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:32.946840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:32.946840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:32.946840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:32.946840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:32.946840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:32.946840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:32.946840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:32.946840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:32.946840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:32.946840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:32.946840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:32.947727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:32.947727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:32.947727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:32.947727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:32.947727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:32.947727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:32.947727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:32.947727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:32.947727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:32.947727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:32.947727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:32.947727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:32.947727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:32.947727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:32.947727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:32.947727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:32.947727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:32.948617 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:32.948617 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:32.948617 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:32.948617 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:32.949231 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 239/489] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:32.958335 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25303eff-0ad8-4f29-886d-ff035294b910 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:32.968466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:32.999963 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:33.075411 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7b4c8-2168-45db-a546-c76a20ee95e4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:33.103340 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25303eff-0ad8-4f29-886d-ff035294b910 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:33.141905 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc202ff2-885a-44b9-9d4f-db5ba0d7b147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:33.142582 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc202ff2-885a-44b9-9d4f-db5ba0d7b147 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:33.142865 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc202ff2-885a-44b9-9d4f-db5ba0d7b147 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:33.143142 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc202ff2-885a-44b9-9d4f-db5ba0d7b147 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:33.143953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc202ff2-885a-44b9-9d4f-db5ba0d7b147 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:33.144266 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc202ff2-885a-44b9-9d4f-db5ba0d7b147 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:33.146637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc202ff2-885a-44b9-9d4f-db5ba0d7b147 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:33.158413 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a9a8e-a3b5-40ee-b3e0-e80a25aab1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:33.159221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a9a8e-a3b5-40ee-b3e0-e80a25aab1f1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:33.159508 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a9a8e-a3b5-40ee-b3e0-e80a25aab1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:33.159785 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a9a8e-a3b5-40ee-b3e0-e80a25aab1f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:33.164654 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 240/490] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:33.167137 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc202ff2-885a-44b9-9d4f-db5ba0d7b147 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:33.186362 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281a9a8e-a3b5-40ee-b3e0-e80a25aab1f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:33.188089 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281a9a8e-a3b5-40ee-b3e0-e80a25aab1f1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:33.216534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-281a9a8e-a3b5-40ee-b3e0-e80a25aab1f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:33.216534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:33.216534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:33.216534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:33.216534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:33.216534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:33.216534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:33.216534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:33.216534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:33.216534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:33.216534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:33.216534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:33.216534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:33.216534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:33.216534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:33.217422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:33.217422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:33.217422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:33.217422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:33.217422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:33.217422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:33.217422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:33.217422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:33.217422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:33.217422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:33.217422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:33.217422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:33.217422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:33.217422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:33.217422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:33.217422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:33.217422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:33.218242 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:33.218242 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:33.218242 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:33.218242 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:33.218242 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 251/491] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:33.224778 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a9a8e-a3b5-40ee-b3e0-e80a25aab1f1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:33.275151 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:33.311221 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc202ff2-885a-44b9-9d4f-db5ba0d7b147 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:33.348802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9918c4a-d967-41e6-8076-2c355cfb93c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:33.349719 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9918c4a-d967-41e6-8076-2c355cfb93c0 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:33.349915 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9918c4a-d967-41e6-8076-2c355cfb93c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:33.350663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9918c4a-d967-41e6-8076-2c355cfb93c0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:33.358456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9918c4a-d967-41e6-8076-2c355cfb93c0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:33.359927 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9918c4a-d967-41e6-8076-2c355cfb93c0 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:33.384047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-e9918c4a-d967-41e6-8076-2c355cfb93c0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 19:48:33.384047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:33.384047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:33.384047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:33.384047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:33.384047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:33.384047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:33.384047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:33.384047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:33.384047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:33.384047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:33.384047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:33.384047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:48:33.384047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:48:33.384047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:48:33.384047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:48:33.386293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:33.386293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:33.386293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:48:33.386293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:33.386293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:48:33.386293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:33.386293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:33.386293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:33.386293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:33.386293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:33.386293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:33.386293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:33.386293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:33.386293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:33.386293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:33.386293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:33.386293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:33.386293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:33.387169 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:33.387169 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:33.387169 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:48:33.387169 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:48:33.387169 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:48:33.387169 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:48:33.387169 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:48:33.387169 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:48:33.387169 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 19:48:33.387169 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:33.387169 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 241/492] 10.222.0.44 () {56 vars in 1093 bytes} [Thu Jun 23 19:48:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:48:33.393360 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9918c4a-d967-41e6-8076-2c355cfb93c0 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:33.470766 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a9a8e-a3b5-40ee-b3e0-e80a25aab1f1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:33.497932 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:33.510976 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09931ce9-7073-4302-8239-b14574e8d997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:33.511931 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09931ce9-7073-4302-8239-b14574e8d997 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:33.512457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09931ce9-7073-4302-8239-b14574e8d997 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:33.512938 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09931ce9-7073-4302-8239-b14574e8d997 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:33.513901 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09931ce9-7073-4302-8239-b14574e8d997 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:33.514425 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09931ce9-7073-4302-8239-b14574e8d997 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:33.515991 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09931ce9-7073-4302-8239-b14574e8d997 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:33.532144 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 252/493] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:33.542669 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09931ce9-7073-4302-8239-b14574e8d997 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:33.601222 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9918c4a-d967-41e6-8076-2c355cfb93c0 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:33.651013 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:33.672338 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79ede1d-f470-4f21-9395-6cd033127f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:33.672338 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ede1d-f470-4f21-9395-6cd033127f36 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:33.672338 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ede1d-f470-4f21-9395-6cd033127f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:33.672338 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ede1d-f470-4f21-9395-6cd033127f36 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:33.672737 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79ede1d-f470-4f21-9395-6cd033127f36 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:33.672737 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79ede1d-f470-4f21-9395-6cd033127f36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:33.673878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79ede1d-f470-4f21-9395-6cd033127f36 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:33.687039 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09931ce9-7073-4302-8239-b14574e8d997 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:33.717166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 242/494] 10.222.0.44 () {56 vars in 1133 bytes} [Thu Jun 23 19:48:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:33.722659 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79ede1d-f470-4f21-9395-6cd033127f36 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:33.793635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3d6e4-b164-404b-9779-bbb71f161263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:33.794340 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3d6e4-b164-404b-9779-bbb71f161263 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:33.794617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3d6e4-b164-404b-9779-bbb71f161263 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:33.794886 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3d6e4-b164-404b-9779-bbb71f161263 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:33.809128 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a3d6e4-b164-404b-9779-bbb71f161263 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:33.810608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a3d6e4-b164-404b-9779-bbb71f161263 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:33.832662 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79ede1d-f470-4f21-9395-6cd033127f36 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:33.846074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-f5a3d6e4-b164-404b-9779-bbb71f161263 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 19:48:33.846074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:33.846074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:33.846074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:33.846074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:33.846074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:33.846074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:33.846074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:33.846074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:33.846074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:33.846074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:33.846074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:33.846074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:48:33.846074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:48:33.846074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:48:33.846074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:48:33.846994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:33.846994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:33.846994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:48:33.846994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:33.846994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:48:33.846994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:33.846994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:33.846994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:33.846994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:33.846994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:33.846994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:33.846994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:33.846994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:33.846994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:33.846994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:33.846994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:33.846994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:33.846994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:33.848345 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:33.848345 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:33.848345 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:48:33.848345 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:48:33.848345 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:48:33.848345 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:48:33.848345 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:48:33.848345 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:48:33.848345 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 19:48:33.848345 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:33.848345 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 253/495] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:33.861369 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3d6e4-b164-404b-9779-bbb71f161263 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:33.878076 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030736a-90ce-4dcd-8217-0e2345528cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:33.878734 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030736a-90ce-4dcd-8217-0e2345528cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:33.879014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030736a-90ce-4dcd-8217-0e2345528cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:33.879285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030736a-90ce-4dcd-8217-0e2345528cf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:33.939256 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:33.960888 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4030736a-90ce-4dcd-8217-0e2345528cf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:33.962393 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4030736a-90ce-4dcd-8217-0e2345528cf8 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:33.983651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:34.040167 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3d6e4-b164-404b-9779-bbb71f161263 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:34.075759 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117d3d4e-5310-4f51-b4c0-9690f7cf51e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:34.076463 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d3d4e-5310-4f51-b4c0-9690f7cf51e8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:34.077065 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d3d4e-5310-4f51-b4c0-9690f7cf51e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:34.077065 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117d3d4e-5310-4f51-b4c0-9690f7cf51e8 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:34.077854 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d3d4e-5310-4f51-b4c0-9690f7cf51e8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:34.078172 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d3d4e-5310-4f51-b4c0-9690f7cf51e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:34.079675 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117d3d4e-5310-4f51-b4c0-9690f7cf51e8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:34.092479 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 243/496] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:48:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:48:34.094009 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3cdc4-43f0-421a-8511-b4c8b7622579 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:34.094468 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3cdc4-43f0-421a-8511-b4c8b7622579 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:34.094889 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3cdc4-43f0-421a-8511-b4c8b7622579 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:34.096316 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 244/497] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:34.103190 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3cdc4-43f0-421a-8511-b4c8b7622579 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:34.131352 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 254/498] 10.222.0.44 () {56 vars in 1137 bytes} [Thu Jun 23 19:48:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:34.133382 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a1fb4-f5d7-4148-b269-3a97c9ba2a8e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:34.133821 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a1fb4-f5d7-4148-b269-3a97c9ba2a8e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:34.134533 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a1fb4-f5d7-4148-b269-3a97c9ba2a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:34.155663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d20a1fb4-f5d7-4148-b269-3a97c9ba2a8e None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:34.156911 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d20a1fb4-f5d7-4148-b269-3a97c9ba2a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:34.201043 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:34.227675 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3cdc4-43f0-421a-8511-b4c8b7622579 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:34.240876 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 255/499] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:34.249204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a1fb4-f5d7-4148-b269-3a97c9ba2a8e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:34.277818 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58061442-4cfc-48cd-9347-5e905ad8c1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:34.278641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58061442-4cfc-48cd-9347-5e905ad8c1a2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:34.278928 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58061442-4cfc-48cd-9347-5e905ad8c1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:34.279229 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58061442-4cfc-48cd-9347-5e905ad8c1a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:34.283652 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:34.290375 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58061442-4cfc-48cd-9347-5e905ad8c1a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:34.293173 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58061442-4cfc-48cd-9347-5e905ad8c1a2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:34.310030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-58061442-4cfc-48cd-9347-5e905ad8c1a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:34.310030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:34.310030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:34.310030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:34.310030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:34.310030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:34.310030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:34.310030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:34.310030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:34.310030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:34.310030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:34.310030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:34.310030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:34.310030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:34.310030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:34.311097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:34.311097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:34.311097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:34.311097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:34.311097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:34.311097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:34.311097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:34.311097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:34.311097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:34.311097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:34.311097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:34.311097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:34.311097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:34.311097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:34.311097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:34.311097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:34.311097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:34.311969 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:34.311969 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:34.311969 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:34.311969 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:34.311969 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 245/500] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:34.312604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05475107-1467-4df8-9ba9-f04730b2855e None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:34.313036 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05475107-1467-4df8-9ba9-f04730b2855e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:34.313321 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05475107-1467-4df8-9ba9-f04730b2855e None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:34.316430 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 246/501] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:34.316430 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05475107-1467-4df8-9ba9-f04730b2855e None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:34.365313 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a1fb4-f5d7-4148-b269-3a97c9ba2a8e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:34.401882 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9e23d-9c3f-4ffc-a361-a99286097e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:34.402838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9e23d-9c3f-4ffc-a361-a99286097e45 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:34.403336 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9e23d-9c3f-4ffc-a361-a99286097e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:34.403518 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9e23d-9c3f-4ffc-a361-a99286097e45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:34.428366 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05475107-1467-4df8-9ba9-f04730b2855e None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:34.458671 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:34.463434 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5801f06-2cba-4fc2-88aa-c41d67263d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:34.464223 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5801f06-2cba-4fc2-88aa-c41d67263d8e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:34.464523 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5801f06-2cba-4fc2-88aa-c41d67263d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:34.464803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5801f06-2cba-4fc2-88aa-c41d67263d8e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:34.475185 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5801f06-2cba-4fc2-88aa-c41d67263d8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:34.476504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d9e23d-9c3f-4ffc-a361-a99286097e45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:34.477984 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d9e23d-9c3f-4ffc-a361-a99286097e45 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:34.482831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-c5801f06-2cba-4fc2-88aa-c41d67263d8e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:34.482831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:34.482831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:34.482831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:34.482831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:34.482831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:34.482831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:34.482831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:34.482831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:34.482831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:34.482831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:34.482831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:34.482831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:34.482831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:34.482831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:34.482831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:34.483776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:34.483776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:34.483776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:34.483776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:34.483776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:34.483776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:34.483776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:34.483776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:34.483776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:34.483776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:34.483776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:34.483776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:34.483776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:34.483776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:34.483776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:34.483776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:34.484641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:34.484641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:34.484641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:34.484641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:34.484641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:34.484641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:34.484641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:34.484641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:34.484641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:34.484641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:34.484641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:34.484641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:34.484641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 247/502] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jun 23 19:48:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:34.485863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5801f06-2cba-4fc2-88aa-c41d67263d8e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:34.508515 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:34.576027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 256/503] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:48:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:48:34.577338 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898947c7-c081-4c8c-9e2a-a08778a851dd None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:34.577635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898947c7-c081-4c8c-9e2a-a08778a851dd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:34.577904 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898947c7-c081-4c8c-9e2a-a08778a851dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:34.587005 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5801f06-2cba-4fc2-88aa-c41d67263d8e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:34.593754 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-898947c7-c081-4c8c-9e2a-a08778a851dd None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:34.594940 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-898947c7-c081-4c8c-9e2a-a08778a851dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:34.620385 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddcaabb-0092-4843-9c21-d46b8f85cf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:34.620981 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddcaabb-0092-4843-9c21-d46b8f85cf7d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:34.621268 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddcaabb-0092-4843-9c21-d46b8f85cf7d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:34.621546 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddcaabb-0092-4843-9c21-d46b8f85cf7d None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:34.622271 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddcaabb-0092-4843-9c21-d46b8f85cf7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:34.622578 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddcaabb-0092-4843-9c21-d46b8f85cf7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:34.623977 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddcaabb-0092-4843-9c21-d46b8f85cf7d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:34.635260 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 248/504] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:34.636573 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda8d14-3bc5-41f6-99af-308819f23789 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:34.636861 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda8d14-3bc5-41f6-99af-308819f23789 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:34.637139 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda8d14-3bc5-41f6-99af-308819f23789 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:34.638366 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 249/505] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:34.639054 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda8d14-3bc5-41f6-99af-308819f23789 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:34.664188 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 257/506] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:34.665064 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898947c7-c081-4c8c-9e2a-a08778a851dd None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:34.701253 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:34.727173 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:34.745704 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda8d14-3bc5-41f6-99af-308819f23789 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:34.764288 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898947c7-c081-4c8c-9e2a-a08778a851dd None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:34.779036 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25c2db-3e02-41f2-b6ac-10fad1698363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:34.779691 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25c2db-3e02-41f2-b6ac-10fad1698363 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:34.779968 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25c2db-3e02-41f2-b6ac-10fad1698363 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:34.780283 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25c2db-3e02-41f2-b6ac-10fad1698363 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:34.781060 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac25c2db-3e02-41f2-b6ac-10fad1698363 None admin] RBAC: Adding query filter params (name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:34.781363 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac25c2db-3e02-41f2-b6ac-10fad1698363 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:34.782730 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac25c2db-3e02-41f2-b6ac-10fad1698363 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:34.790572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 250/507] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jun 23 19:48:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:34.791356 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25c2db-3e02-41f2-b6ac-10fad1698363 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:34.798653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27ad04-9d33-4289-8c63-45ea323bc384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:34.799832 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27ad04-9d33-4289-8c63-45ea323bc384 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:34.800114 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27ad04-9d33-4289-8c63-45ea323bc384 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:34.800415 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27ad04-9d33-4289-8c63-45ea323bc384 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:34.810365 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b27ad04-9d33-4289-8c63-45ea323bc384 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:34.811869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b27ad04-9d33-4289-8c63-45ea323bc384 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:34.829441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-3b27ad04-9d33-4289-8c63-45ea323bc384 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:34.829441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:34.829441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:34.829441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:34.829441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:34.829441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:34.829441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:34.829441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:34.829441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:34.829441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:34.829441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:34.829441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:34.829441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:34.829441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:34.829441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:34.830289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:34.830289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:34.830289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:34.830289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:34.830289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:34.830289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:34.830289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:34.830289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:34.830289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:34.830289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:34.830289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:34.830289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:34.830289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:34.830289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:34.830289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:34.830289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:34.830289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:34.831127 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:34.831127 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:34.831127 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:34.831127 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:34.831127 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 258/508] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:34.831779 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574cf6f-0c5d-4dca-a7c4-5dcc5e12b9c5 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:34.832015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574cf6f-0c5d-4dca-a7c4-5dcc5e12b9c5 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:34.832281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1574cf6f-0c5d-4dca-a7c4-5dcc5e12b9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:34.847630 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1574cf6f-0c5d-4dca-a7c4-5dcc5e12b9c5 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:34.848833 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1574cf6f-0c5d-4dca-a7c4-5dcc5e12b9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:34.891646 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25c2db-3e02-41f2-b6ac-10fad1698363 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:34.904539 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:34.913879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 259/509] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:34.914689 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1574cf6f-0c5d-4dca-a7c4-5dcc5e12b9c5 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:34.923572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c1b72-694e-4324-bd56-c4a456eba676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:34.924398 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c1b72-694e-4324-bd56-c4a456eba676 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:34.924683 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c1b72-694e-4324-bd56-c4a456eba676 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:34.924956 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c1b72-694e-4324-bd56-c4a456eba676 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:34.932236 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c1b72-694e-4324-bd56-c4a456eba676 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:34.934552 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c1b72-694e-4324-bd56-c4a456eba676 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:34.951625 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-db4c1b72-694e-4324-bd56-c4a456eba676 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:34.951625 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:34.951625 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:34.951625 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:34.951625 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:34.951625 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:34.951625 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:34.951625 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:34.951625 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:34.951625 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:34.951625 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:34.951625 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:34.951625 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:48:34.951625 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:48:34.951625 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:48:34.952714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:48:34.952714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:34.952714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:34.952714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:34.952714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:34.952714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:34.952714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:34.952714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:34.952714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:34.952714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:34.952714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:34.952714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:34.952714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:34.952714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:34.952714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:34.952714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:34.952714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:34.953560 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:48:34.953560 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:48:34.953560 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:48:34.953560 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:48:34.953560 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:48:34.953560 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:48:34.953560 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:34.953560 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:34.953560 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 251/510] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:48:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:34.954018 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c1b72-694e-4324-bd56-c4a456eba676 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:34.978451 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:35.049988 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1574cf6f-0c5d-4dca-a7c4-5dcc5e12b9c5 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:35.059700 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c1b72-694e-4324-bd56-c4a456eba676 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:35.086825 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e084b-ec92-46a0-860b-93ef33b66434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:35.087611 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e084b-ec92-46a0-860b-93ef33b66434 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:35.087890 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e084b-ec92-46a0-860b-93ef33b66434 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:35.088159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6e084b-ec92-46a0-860b-93ef33b66434 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:35.094055 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c4be30-bc82-4a09-b173-e2d856676dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:35.094845 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4be30-bc82-4a09-b173-e2d856676dcb None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:35.095127 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4be30-bc82-4a09-b173-e2d856676dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:35.095403 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c4be30-bc82-4a09-b173-e2d856676dcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:35.097740 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6e084b-ec92-46a0-860b-93ef33b66434 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:35.099253 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6e084b-ec92-46a0-860b-93ef33b66434 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:35.107317 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c4be30-bc82-4a09-b173-e2d856676dcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:35.108829 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c4be30-bc82-4a09-b173-e2d856676dcb None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:35.117685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-4f6e084b-ec92-46a0-860b-93ef33b66434 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:35.117685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:35.117685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:35.117685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:35.117685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:35.117685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:35.117685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:35.117685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:35.117685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:35.117685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:35.117685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:35.117685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:35.117685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:35.117685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:35.117685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:35.118542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:35.118542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:35.118542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:35.118542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:35.118542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:35.118542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:35.118542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:35.118542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:35.118542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:35.118542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:35.118542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:35.118542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:35.118542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:35.118542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:35.118542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:35.118542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:35.118542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:35.119371 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:35.119371 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:35.119371 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:35.119371 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:35.119371 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 260/511] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:35.119826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e084b-ec92-46a0-860b-93ef33b66434 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:35.131953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-24c4be30-bc82-4a09-b173-e2d856676dcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:35.131953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:35.131953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:35.131953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:35.131953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:35.131953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:35.131953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:35.131953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:35.131953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:35.131953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:35.131953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:35.131953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:35.131953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:35.131953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:35.131953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:35.132864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:35.132864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:35.132864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:35.132864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:35.132864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:35.132864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:35.132864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:35.132864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:35.132864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:35.132864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:35.132864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:35.132864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:35.132864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:35.132864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:35.132864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:35.132864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:35.132864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:35.133699 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:35.133699 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:35.133699 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:35.133699 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:35.133699 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 252/512] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:35.133971 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c4be30-bc82-4a09-b173-e2d856676dcb None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:35.159842 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:35.217056 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:35.222593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6e084b-ec92-46a0-860b-93ef33b66434 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:35.231868 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c4be30-bc82-4a09-b173-e2d856676dcb None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:35.257140 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096fd57-4b4b-47ad-b762-6631ef67e465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:35.257799 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096fd57-4b4b-47ad-b762-6631ef67e465 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:35.258080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096fd57-4b4b-47ad-b762-6631ef67e465 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:35.258353 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096fd57-4b4b-47ad-b762-6631ef67e465 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:35.259064 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b096fd57-4b4b-47ad-b762-6631ef67e465 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:35.259366 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b096fd57-4b4b-47ad-b762-6631ef67e465 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:35.260761 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b096fd57-4b4b-47ad-b762-6631ef67e465 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:35.267795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f720b5f0-6570-448c-8c05-2ef167d7459d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:35.268610 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720b5f0-6570-448c-8c05-2ef167d7459d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:35.268893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720b5f0-6570-448c-8c05-2ef167d7459d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:35.269169 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720b5f0-6570-448c-8c05-2ef167d7459d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:35.272895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 261/513] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:35.273700 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096fd57-4b4b-47ad-b762-6631ef67e465 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:35.283195 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f720b5f0-6570-448c-8c05-2ef167d7459d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:35.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-f720b5f0-6570-448c-8c05-2ef167d7459d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:35.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:35.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:35.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:35.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:35.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:35.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:35.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:35.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:35.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:35.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:35.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:35.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:35.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:35.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:35.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:35.292364 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:35.292364 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:35.292364 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:35.292364 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:35.292364 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:35.292364 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:35.292364 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:35.292364 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:35.292364 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:35.292364 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:35.292364 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:35.292364 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:35.292364 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:35.292364 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:35.292364 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:35.292364 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:35.293189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:35.293189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:35.293189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:35.293189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:35.293189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:35.293189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:35.293189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:35.293189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:35.293189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:35.293189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:35.293189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:35.293189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:35.293189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 253/514] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jun 23 19:48:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:35.293833 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f720b5f0-6570-448c-8c05-2ef167d7459d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:35.374962 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096fd57-4b4b-47ad-b762-6631ef67e465 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:35.375661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:35.390404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f720b5f0-6570-448c-8c05-2ef167d7459d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:35.407849 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11010365-89a3-413a-b38d-44562d601c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:35.408539 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11010365-89a3-413a-b38d-44562d601c0c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:35.408825 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11010365-89a3-413a-b38d-44562d601c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:35.409101 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11010365-89a3-413a-b38d-44562d601c0c None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:35.409935 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11010365-89a3-413a-b38d-44562d601c0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:35.410194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11010365-89a3-413a-b38d-44562d601c0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:35.412247 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11010365-89a3-413a-b38d-44562d601c0c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:35.423297 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a78eb-e2f9-4973-ac43-3591ecbdd685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:35.423949 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a78eb-e2f9-4973-ac43-3591ecbdd685 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:35.424235 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a78eb-e2f9-4973-ac43-3591ecbdd685 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:35.424543 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a78eb-e2f9-4973-ac43-3591ecbdd685 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:35.425265 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a78eb-e2f9-4973-ac43-3591ecbdd685 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:35.425575 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a78eb-e2f9-4973-ac43-3591ecbdd685 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:35.426949 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a78eb-e2f9-4973-ac43-3591ecbdd685 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:35.429550 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 262/515] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:48:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:35.430348 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11010365-89a3-413a-b38d-44562d601c0c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:35.438929 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 254/516] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:35.439750 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a78eb-e2f9-4973-ac43-3591ecbdd685 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:35.451297 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:35.532835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11010365-89a3-413a-b38d-44562d601c0c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:35.538901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a78eb-e2f9-4973-ac43-3591ecbdd685 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:35.565575 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a228e1c-60e1-4724-9670-a67c362ed938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:35.566235 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a228e1c-60e1-4724-9670-a67c362ed938 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:35.566521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a228e1c-60e1-4724-9670-a67c362ed938 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:35.566808 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a228e1c-60e1-4724-9670-a67c362ed938 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:35.567525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a228e1c-60e1-4724-9670-a67c362ed938 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:35.568345 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a228e1c-60e1-4724-9670-a67c362ed938 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:35.569730 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a228e1c-60e1-4724-9670-a67c362ed938 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:35.571857 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfef715-3bfd-4a17-b3ad-5e276140074c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:35.572663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfef715-3bfd-4a17-b3ad-5e276140074c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:35.572946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfef715-3bfd-4a17-b3ad-5e276140074c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:35.573219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfef715-3bfd-4a17-b3ad-5e276140074c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:35.579699 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfef715-3bfd-4a17-b3ad-5e276140074c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:35.580861 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 263/517] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:35.581162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfef715-3bfd-4a17-b3ad-5e276140074c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:35.581640 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a228e1c-60e1-4724-9670-a67c362ed938 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:35.595261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-2bfef715-3bfd-4a17-b3ad-5e276140074c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:35.595261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:35.595261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:35.595261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:35.595261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:35.595261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:35.595261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:35.595261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:35.595261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:35.595261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:35.595261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:35.595261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:35.595261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:48:35.595261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:48:35.595261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:48:35.596105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:48:35.596105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:35.596105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:35.596105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:35.596105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:35.596105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:35.596105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:35.596105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:35.596105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:35.596105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:35.596105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:35.596105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:35.596105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:35.596105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:35.596105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:35.596105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:35.596105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:35.597000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:48:35.597000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:48:35.597000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:48:35.597000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:48:35.597000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:48:35.597000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:48:35.597000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:35.597000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:35.597000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 255/518] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:48:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:35.597645 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfef715-3bfd-4a17-b3ad-5e276140074c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:35.606329 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:35.661394 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:35.683833 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a228e1c-60e1-4724-9670-a67c362ed938 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:35.694667 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfef715-3bfd-4a17-b3ad-5e276140074c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:35.717058 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8863cb4-b790-4c02-9110-4d67a9b17b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:35.717716 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8863cb4-b790-4c02-9110-4d67a9b17b8d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:35.717998 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8863cb4-b790-4c02-9110-4d67a9b17b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:35.718273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8863cb4-b790-4c02-9110-4d67a9b17b8d None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:35.719014 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8863cb4-b790-4c02-9110-4d67a9b17b8d None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:35.719320 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8863cb4-b790-4c02-9110-4d67a9b17b8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:35.720737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8863cb4-b790-4c02-9110-4d67a9b17b8d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:35.727287 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae95a61-fa9e-4043-ac37-ceda01a20907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:35.727951 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae95a61-fa9e-4043-ac37-ceda01a20907 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:35.728229 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae95a61-fa9e-4043-ac37-ceda01a20907 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:35.728528 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae95a61-fa9e-4043-ac37-ceda01a20907 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:35.732945 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 264/519] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jun 23 19:48:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:35.733057 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae95a61-fa9e-4043-ac37-ceda01a20907 None admin] RBAC: Adding query filter params (role.id=5f27c72acb19431fa3223c962bc42938, scope.project.id=155180ba63c446ca889a6eb3f94cc31a, user.id=a63d1d902c024c97a38f7d92f73814d3) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:35.733145 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8863cb4-b790-4c02-9110-4d67a9b17b8d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:35.733225 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae95a61-fa9e-4043-ac37-ceda01a20907 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:35.733225 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae95a61-fa9e-4043-ac37-ceda01a20907 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:35.744816 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 256/520] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:48:35 2022] GET /identity/v3/role_assignments?user.id=a63d1d902c024c97a38f7d92f73814d3&scope.project.id=155180ba63c446ca889a6eb3f94cc31a&role.id=5f27c72acb19431fa3223c962bc42938 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:35.744816 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae95a61-fa9e-4043-ac37-ceda01a20907 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:35.815835 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:35.854380 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8863cb4-b790-4c02-9110-4d67a9b17b8d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:35.854594 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae95a61-fa9e-4043-ac37-ceda01a20907 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:35.890275 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5603e9-2780-496a-b3a4-d75a005f2033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:35.891351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5603e9-2780-496a-b3a4-d75a005f2033 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:35.891640 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5603e9-2780-496a-b3a4-d75a005f2033 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:35.891918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5603e9-2780-496a-b3a4-d75a005f2033 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:35.899958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34bd7c2-b917-46f2-8905-b9804149d5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:35.900807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34bd7c2-b917-46f2-8905-b9804149d5ac None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:35.901108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34bd7c2-b917-46f2-8905-b9804149d5ac None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:35.901361 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34bd7c2-b917-46f2-8905-b9804149d5ac None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:35.901828 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5603e9-2780-496a-b3a4-d75a005f2033 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:35.902069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34bd7c2-b917-46f2-8905-b9804149d5ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:35.903316 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5603e9-2780-496a-b3a4-d75a005f2033 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:35.905384 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34bd7c2-b917-46f2-8905-b9804149d5ac None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:35.915488 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:35.918658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-7e5603e9-2780-496a-b3a4-d75a005f2033 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 19:48:35.918658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:35.918658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:35.918658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:35.918658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:35.918658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:35.918658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:35.918658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:35.918658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:35.918658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:35.918658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:35.918658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:35.918658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:48:35.918658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:48:35.918658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:48:35.918658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:48:35.919769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:35.919769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:35.919769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:48:35.919769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:35.919769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:48:35.919769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:35.919769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:35.919769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:35.919769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:35.919769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:35.919769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:35.919769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:35.919769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:35.919769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:35.919769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:35.919769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:35.919769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:35.919769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:35.921235 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:35.921235 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:35.921235 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:48:35.921235 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:48:35.921235 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:48:35.921235 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:48:35.921235 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:48:35.921235 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:48:35.921235 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 19:48:35.921235 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:35.921235 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 257/521] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:35.921235 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5603e9-2780-496a-b3a4-d75a005f2033 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:36.002391 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 265/522] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jun 23 19:48:35 2022] POST /identity/v3/users => generated 247 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:48:36.003034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34bd7c2-b917-46f2-8905-b9804149d5ac None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:36.034829 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5603e9-2780-496a-b3a4-d75a005f2033 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:36.080725 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:36.089687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84353bf0-caf2-44fd-baba-d449bd6fd866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:36.090391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84353bf0-caf2-44fd-baba-d449bd6fd866 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:36.090678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84353bf0-caf2-44fd-baba-d449bd6fd866 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:36.090956 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84353bf0-caf2-44fd-baba-d449bd6fd866 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:36.092512 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84353bf0-caf2-44fd-baba-d449bd6fd866 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:36.092856 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84353bf0-caf2-44fd-baba-d449bd6fd866 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:36.094697 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84353bf0-caf2-44fd-baba-d449bd6fd866 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:36.122512 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 258/523] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:48:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:36.123941 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84353bf0-caf2-44fd-baba-d449bd6fd866 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:36.143667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34bd7c2-b917-46f2-8905-b9804149d5ac None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:36.194001 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35139042-04ab-429d-9737-b25ca093ba52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:36.194722 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35139042-04ab-429d-9737-b25ca093ba52 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:36.194996 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35139042-04ab-429d-9737-b25ca093ba52 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:36.195257 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35139042-04ab-429d-9737-b25ca093ba52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:36.209593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35139042-04ab-429d-9737-b25ca093ba52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:36.211130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35139042-04ab-429d-9737-b25ca093ba52 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:36.232322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-35139042-04ab-429d-9737-b25ca093ba52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:36.232322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:36.232322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:36.232322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:36.232322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:36.232322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:36.232322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:36.232322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:36.232322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:36.232322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:36.232322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:36.232322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:36.232322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:36.232322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:36.232322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:36.233226 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:36.233226 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:36.233226 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:36.233226 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:36.233226 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:36.233226 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:36.233226 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:36.233226 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:36.233226 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:36.233226 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:36.233226 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:36.233226 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:36.233226 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:36.233226 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:36.233226 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:36.233226 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:36.233226 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:36.234109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:36.234109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:36.234109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:36.234109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:36.236505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 266/524] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:36.237741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35139042-04ab-429d-9737-b25ca093ba52 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:36.244413 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84353bf0-caf2-44fd-baba-d449bd6fd866 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:36.283225 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193771e6-b351-441c-b467-1d3a6b14a36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:36.283894 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193771e6-b351-441c-b467-1d3a6b14a36f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:36.284172 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193771e6-b351-441c-b467-1d3a6b14a36f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:36.284501 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193771e6-b351-441c-b467-1d3a6b14a36f None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:36.285336 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193771e6-b351-441c-b467-1d3a6b14a36f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:36.285638 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193771e6-b351-441c-b467-1d3a6b14a36f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:36.287638 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193771e6-b351-441c-b467-1d3a6b14a36f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:36.301550 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:36.305727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 259/525] 10.222.0.44 () {56 vars in 1137 bytes} [Thu Jun 23 19:48:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:36.306563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193771e6-b351-441c-b467-1d3a6b14a36f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:36.351935 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35139042-04ab-429d-9737-b25ca093ba52 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:36.366769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:36.398447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d2cdd-70b5-4d1c-a234-aaa3b3919c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:36.399824 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d2cdd-70b5-4d1c-a234-aaa3b3919c60 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:36.400417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d2cdd-70b5-4d1c-a234-aaa3b3919c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:36.400894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d2cdd-70b5-4d1c-a234-aaa3b3919c60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:36.402011 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d2cdd-70b5-4d1c-a234-aaa3b3919c60 None admin] RBAC: Adding query filter params (role.id=5f27c72acb19431fa3223c962bc42938, scope.project.id=155180ba63c446ca889a6eb3f94cc31a, user.id=1dd72f952df34ad8853fe84673a41f45) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:36.402471 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d2cdd-70b5-4d1c-a234-aaa3b3919c60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:36.404848 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d2cdd-70b5-4d1c-a234-aaa3b3919c60 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:36.412618 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 267/526] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:48:36 2022] GET /identity/v3/role_assignments?user.id=1dd72f952df34ad8853fe84673a41f45&scope.project.id=155180ba63c446ca889a6eb3f94cc31a&role.id=5f27c72acb19431fa3223c962bc42938 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:36.414659 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d2cdd-70b5-4d1c-a234-aaa3b3919c60 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:36.424038 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193771e6-b351-441c-b467-1d3a6b14a36f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:36.468205 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2900bf-f299-4f8a-a701-604cae234093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:36.468205 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2900bf-f299-4f8a-a701-604cae234093 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:36.468205 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2900bf-f299-4f8a-a701-604cae234093 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:36.468205 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2900bf-f299-4f8a-a701-604cae234093 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:36.469249 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2900bf-f299-4f8a-a701-604cae234093 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:36.469249 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2900bf-f299-4f8a-a701-604cae234093 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:36.469249 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2900bf-f299-4f8a-a701-604cae234093 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:36.495168 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 260/527] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:36.500757 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2900bf-f299-4f8a-a701-604cae234093 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:36.616533 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d2cdd-70b5-4d1c-a234-aaa3b3919c60 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:36.631903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:36.657630 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0bdb5-bedd-4bc9-845d-614a9be0b5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:36.658289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0bdb5-bedd-4bc9-845d-614a9be0b5b1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:36.658575 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0bdb5-bedd-4bc9-845d-614a9be0b5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:36.658856 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0bdb5-bedd-4bc9-845d-614a9be0b5b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:36.660945 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2900bf-f299-4f8a-a701-604cae234093 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:36.674296 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:36.701706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b52bff-fcdf-4a98-b877-0c13664e7fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:36.703195 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b52bff-fcdf-4a98-b877-0c13664e7fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:36.703639 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b52bff-fcdf-4a98-b877-0c13664e7fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:36.704202 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b52bff-fcdf-4a98-b877-0c13664e7fb0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:36.714282 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b52bff-fcdf-4a98-b877-0c13664e7fb0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:36.716614 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b52bff-fcdf-4a98-b877-0c13664e7fb0 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:36.731384 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0bdb5-bedd-4bc9-845d-614a9be0b5b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:36.733017 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0bdb5-bedd-4bc9-845d-614a9be0b5b1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:36.735186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-77b52bff-fcdf-4a98-b877-0c13664e7fb0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 19:48:36.735186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:36.735186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:36.735186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:36.735186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:36.735186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:36.735186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:36.735186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:36.735186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:36.735186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:36.735186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:36.735186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:36.735186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:48:36.735186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:48:36.735186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:48:36.735186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:48:36.736138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:36.736138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:36.736138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:48:36.736138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:36.736138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:48:36.736138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:36.736138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:36.736138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:36.736138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:36.736138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:36.736138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:36.736138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:36.736138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:36.736138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:36.736138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:36.736138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:36.736138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:36.736138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:36.737312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:36.737312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:36.737312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:48:36.737312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:48:36.737312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:48:36.737312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:48:36.737312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:48:36.737312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:48:36.737312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 19:48:36.737312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:36.739269 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 261/528] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jun 23 19:48:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:48:36.747331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b52bff-fcdf-4a98-b877-0c13664e7fb0 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:36.864510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b52bff-fcdf-4a98-b877-0c13664e7fb0 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:36.905522 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932df269-462d-45fd-a919-803c8685f349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:36.906251 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932df269-462d-45fd-a919-803c8685f349 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:36.906542 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932df269-462d-45fd-a919-803c8685f349 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:36.906819 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932df269-462d-45fd-a919-803c8685f349 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:36.907673 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932df269-462d-45fd-a919-803c8685f349 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:36.907985 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932df269-462d-45fd-a919-803c8685f349 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:36.910110 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932df269-462d-45fd-a919-803c8685f349 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:36.943316 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 262/529] 10.222.0.44 () {56 vars in 1139 bytes} [Thu Jun 23 19:48:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:36.952560 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 268/530] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:48:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:48:36.954065 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932df269-462d-45fd-a919-803c8685f349 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:36.959152 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0bdb5-bedd-4bc9-845d-614a9be0b5b1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:36.980803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:37.003922 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:37.096142 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932df269-462d-45fd-a919-803c8685f349 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:37.113970 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0bdb5-bedd-4bc9-845d-614a9be0b5b1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:37.137071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b913c9e-f49e-4a4f-91b8-2276a100e028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:37.137725 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b913c9e-f49e-4a4f-91b8-2276a100e028 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:37.138010 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b913c9e-f49e-4a4f-91b8-2276a100e028 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:37.138287 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b913c9e-f49e-4a4f-91b8-2276a100e028 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:37.186024 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb8851-eab3-41a3-9f3b-6830bf8a7659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:37.186821 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb8851-eab3-41a3-9f3b-6830bf8a7659 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:37.187106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb8851-eab3-41a3-9f3b-6830bf8a7659 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:37.187379 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb8851-eab3-41a3-9f3b-6830bf8a7659 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:37.213299 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb8851-eab3-41a3-9f3b-6830bf8a7659 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:37.214790 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb8851-eab3-41a3-9f3b-6830bf8a7659 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:37.245294 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-d0cb8851-eab3-41a3-9f3b-6830bf8a7659 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:37.245294 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:37.245294 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:37.245294 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:37.245294 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:37.245294 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:37.245294 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:37.245294 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:37.245294 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:37.245294 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:37.245294 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:37.245294 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:37.245294 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:37.245294 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:37.245294 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:37.246213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:37.246213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:37.246213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:37.246213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:37.246213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:37.246213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:37.246213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:37.246213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:37.246213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:37.246213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:37.246213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:37.246213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:37.246213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:37.246213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:37.246213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:37.246213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:37.246213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:37.247051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:37.247051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:37.247051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:37.247051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:37.247051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 269/531] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:37.254693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb8851-eab3-41a3-9f3b-6830bf8a7659 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:37.272063 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b913c9e-f49e-4a4f-91b8-2276a100e028 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:37.273581 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b913c9e-f49e-4a4f-91b8-2276a100e028 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:37.301912 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:37.350877 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:37.423192 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 263/532] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:48:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 19:48:37.425071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb8851-eab3-41a3-9f3b-6830bf8a7659 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:37.428916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b913c9e-f49e-4a4f-91b8-2276a100e028 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:37.486222 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a058ade-bb85-4000-8ec9-959090d063ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:37.486890 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a058ade-bb85-4000-8ec9-959090d063ac None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:37.487177 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a058ade-bb85-4000-8ec9-959090d063ac None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:37.487684 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a058ade-bb85-4000-8ec9-959090d063ac None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:37.488431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a058ade-bb85-4000-8ec9-959090d063ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:37.488740 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a058ade-bb85-4000-8ec9-959090d063ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:37.490698 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a058ade-bb85-4000-8ec9-959090d063ac None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:37.513320 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 270/533] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:37.518630 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a058ade-bb85-4000-8ec9-959090d063ac None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:37.637942 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a058ade-bb85-4000-8ec9-959090d063ac None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:37.642369 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b913c9e-f49e-4a4f-91b8-2276a100e028 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:37.673428 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:37.694162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:37.696329 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac5f95-6ab9-4ad9-8944-be10e6e23c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:37.697135 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac5f95-6ab9-4ad9-8944-be10e6e23c94 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:37.697419 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac5f95-6ab9-4ad9-8944-be10e6e23c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:37.697694 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac5f95-6ab9-4ad9-8944-be10e6e23c94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:37.707423 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ac5f95-6ab9-4ad9-8944-be10e6e23c94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:37.708975 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ac5f95-6ab9-4ad9-8944-be10e6e23c94 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:37.714134 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4332d-db55-4052-9847-0943bd31c35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:37.714931 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4332d-db55-4052-9847-0943bd31c35e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:37.715217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4332d-db55-4052-9847-0943bd31c35e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:37.715493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4332d-db55-4052-9847-0943bd31c35e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:37.734807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-05ac5f95-6ab9-4ad9-8944-be10e6e23c94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:37.734807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:37.734807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:37.734807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:37.734807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:37.734807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:37.734807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:37.734807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:37.734807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:37.734807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:37.734807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:37.734807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:37.734807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:48:37.734807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:48:37.734807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:48:37.735714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:48:37.735714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:37.735714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:37.735714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:37.735714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:37.735714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:37.735714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:37.735714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:37.735714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:37.735714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:37.735714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:37.735714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:37.735714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:37.735714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:37.735714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:37.735714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:37.735714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:37.736579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:48:37.736579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:48:37.736579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:48:37.736579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:48:37.736579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:48:37.736579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:48:37.736579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:37.736579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:37.740327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 271/534] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:48:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:37.740459 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4332d-db55-4052-9847-0943bd31c35e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:37.741497 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4332d-db55-4052-9847-0943bd31c35e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:37.744257 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac5f95-6ab9-4ad9-8944-be10e6e23c94 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:37.768577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-81f4332d-db55-4052-9847-0943bd31c35e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:37.768577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:37.768577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:37.768577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:37.768577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:37.768577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:37.768577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:37.768577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:37.768577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:37.768577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:37.768577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:37.768577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:37.768577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:37.768577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:37.768577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:37.769539 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:37.769539 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:37.769539 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:37.769539 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:37.769539 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:37.769539 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:37.769539 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:37.769539 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:37.769539 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:37.769539 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:37.769539 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:37.769539 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:37.769539 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:37.769539 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:37.769539 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:37.769539 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:37.769539 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:37.770380 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:37.770380 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:37.770380 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:37.770380 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:37.777319 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 264/535] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:48:37.779219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4332d-db55-4052-9847-0943bd31c35e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:37.939439 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4332d-db55-4052-9847-0943bd31c35e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:37.941207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac5f95-6ab9-4ad9-8944-be10e6e23c94 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:37.976121 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd7dd9-ead6-4ffc-9219-f33e49f40eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:37.976821 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd7dd9-ead6-4ffc-9219-f33e49f40eeb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:37.977105 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd7dd9-ead6-4ffc-9219-f33e49f40eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:37.977351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567de5ef-c67b-4824-8d5a-24187126df14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:37.977495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd7dd9-ead6-4ffc-9219-f33e49f40eeb None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:37.978016 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567de5ef-c67b-4824-8d5a-24187126df14 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:37.978139 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd7dd9-ead6-4ffc-9219-f33e49f40eeb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:37.978310 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567de5ef-c67b-4824-8d5a-24187126df14 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:37.978470 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd7dd9-ead6-4ffc-9219-f33e49f40eeb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:37.978580 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567de5ef-c67b-4824-8d5a-24187126df14 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:37.979276 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567de5ef-c67b-4824-8d5a-24187126df14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:37.979582 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567de5ef-c67b-4824-8d5a-24187126df14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:37.979905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd7dd9-ead6-4ffc-9219-f33e49f40eeb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:37.980994 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567de5ef-c67b-4824-8d5a-24187126df14 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:37.994474 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 272/536] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:48:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:37.995861 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab3e82-b631-4de8-9584-67d82e0926ef None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:37.996153 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab3e82-b631-4de8-9584-67d82e0926ef None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:37.996455 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab3e82-b631-4de8-9584-67d82e0926ef None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:38.001662 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 265/538] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:38.001792 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 273/538] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:38.002526 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567de5ef-c67b-4824-8d5a-24187126df14 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:38.003079 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ded24-5a83-4b38-a079-8a5f7a45e19b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:38.003375 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ded24-5a83-4b38-a079-8a5f7a45e19b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:38.003653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ded24-5a83-4b38-a079-8a5f7a45e19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:38.020202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-341ded24-5a83-4b38-a079-8a5f7a45e19b None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:38.021841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-341ded24-5a83-4b38-a079-8a5f7a45e19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:38.044668 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:38.060239 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:38.104510 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 274/539] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 19:48:38.105475 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ded24-5a83-4b38-a079-8a5f7a45e19b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:38.134229 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567de5ef-c67b-4824-8d5a-24187126df14 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:38.172651 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ef9ff-49f2-47f8-b655-75ef9c9ddbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:38.173308 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ef9ff-49f2-47f8-b655-75ef9c9ddbc2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:38.173733 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ef9ff-49f2-47f8-b655-75ef9c9ddbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:38.173963 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ef9ff-49f2-47f8-b655-75ef9c9ddbc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:38.175072 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ef9ff-49f2-47f8-b655-75ef9c9ddbc2 None admin] RBAC: Adding query filter params (role.id=5f27c72acb19431fa3223c962bc42938, scope.project.id=155180ba63c446ca889a6eb3f94cc31a, user.id=21ae3b8971b44d928e9f6c0539858636) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:38.175206 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ef9ff-49f2-47f8-b655-75ef9c9ddbc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:38.176664 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ef9ff-49f2-47f8-b655-75ef9c9ddbc2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:38.183580 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 266/540] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:48:38 2022] GET /identity/v3/role_assignments?user.id=21ae3b8971b44d928e9f6c0539858636&scope.project.id=155180ba63c446ca889a6eb3f94cc31a&role.id=5f27c72acb19431fa3223c962bc42938 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:38.184478 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ef9ff-49f2-47f8-b655-75ef9c9ddbc2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:38.261571 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ded24-5a83-4b38-a079-8a5f7a45e19b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:38.343703 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02991f72-42bd-44cd-8f00-a56ed62183d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:38.344554 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02991f72-42bd-44cd-8f00-a56ed62183d1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:38.344839 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02991f72-42bd-44cd-8f00-a56ed62183d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:38.345112 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02991f72-42bd-44cd-8f00-a56ed62183d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:38.345930 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ef9ff-49f2-47f8-b655-75ef9c9ddbc2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:38.347600 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:38.366716 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02991f72-42bd-44cd-8f00-a56ed62183d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:38.368214 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02991f72-42bd-44cd-8f00-a56ed62183d1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:38.393023 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-02991f72-42bd-44cd-8f00-a56ed62183d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:38.393023 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:38.393023 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:38.393023 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:38.393023 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:38.393023 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:38.393023 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:38.393023 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:38.393023 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:38.393023 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:38.393023 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:38.393023 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:38.393023 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:48:38.393023 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:48:38.393023 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:48:38.393919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:48:38.393919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:38.393919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:38.393919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:38.393919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:38.393919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:38.393919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:38.393919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:38.393919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:38.393919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:38.393919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:38.393919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:38.393919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:38.393919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:38.393919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:38.393919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:38.393919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:38.394737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:48:38.394737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:48:38.394737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:48:38.394737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:48:38.394737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:48:38.394737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:48:38.394737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:38.394737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:38.394737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 275/541] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:48:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:38.412580 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02991f72-42bd-44cd-8f00-a56ed62183d1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:38.426113 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a627d34b-f433-49b9-9fd0-91446385c0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:38.426929 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627d34b-f433-49b9-9fd0-91446385c0a3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:38.427776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627d34b-f433-49b9-9fd0-91446385c0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:38.428066 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627d34b-f433-49b9-9fd0-91446385c0a3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:38.454882 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a627d34b-f433-49b9-9fd0-91446385c0a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:38.462786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-a627d34b-f433-49b9-9fd0-91446385c0a3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:38.462786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:38.462786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:38.462786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:38.462786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:38.462786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:38.462786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:38.462786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:38.462786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:38.462786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:38.462786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:38.462786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:38.462786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:38.462786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:38.462786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:38.462786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:38.464886 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:38.464886 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:38.464886 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:38.464886 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:38.464886 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:38.464886 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:38.464886 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:38.464886 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:38.464886 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:38.464886 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:38.464886 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:38.464886 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:38.464886 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:38.464886 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:38.464886 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:38.464886 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:38.466921 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:38.466921 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:38.466921 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:38.466921 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:38.466921 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:38.466921 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:38.466921 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:38.466921 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:38.466921 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:38.466921 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:38.466921 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:38.466921 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:38.466921 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 267/542] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jun 23 19:48:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:38.472751 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a627d34b-f433-49b9-9fd0-91446385c0a3 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:38.478686 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:38.569847 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02991f72-42bd-44cd-8f00-a56ed62183d1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:38.607733 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c3fdc1-a1f3-45d8-9c75-3b6835e1ed12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:38.608772 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3fdc1-a1f3-45d8-9c75-3b6835e1ed12 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:38.609233 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3fdc1-a1f3-45d8-9c75-3b6835e1ed12 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:38.609354 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a627d34b-f433-49b9-9fd0-91446385c0a3 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:38.609808 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3fdc1-a1f3-45d8-9c75-3b6835e1ed12 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:38.610877 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c3fdc1-a1f3-45d8-9c75-3b6835e1ed12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:38.611361 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c3fdc1-a1f3-45d8-9c75-3b6835e1ed12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:38.613098 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c3fdc1-a1f3-45d8-9c75-3b6835e1ed12 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:38.627305 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 276/543] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:48:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:38.628870 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a1fc0-85b9-4db7-92af-5c5c35c26bb3 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:38.629339 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a1fc0-85b9-4db7-92af-5c5c35c26bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:38.629806 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a1fc0-85b9-4db7-92af-5c5c35c26bb3 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:38.631389 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 277/544] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:38.636360 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a1fc0-85b9-4db7-92af-5c5c35c26bb3 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:38.645516 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c86bd2e-6a2b-49d6-afdb-e4431aad667c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:38.646809 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86bd2e-6a2b-49d6-afdb-e4431aad667c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:38.647168 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86bd2e-6a2b-49d6-afdb-e4431aad667c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:38.647519 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86bd2e-6a2b-49d6-afdb-e4431aad667c None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:38.648377 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c86bd2e-6a2b-49d6-afdb-e4431aad667c None admin] RBAC: Adding query filter params (name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:38.648759 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c86bd2e-6a2b-49d6-afdb-e4431aad667c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:38.650228 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c86bd2e-6a2b-49d6-afdb-e4431aad667c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:38.658954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 268/545] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jun 23 19:48:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:38.664789 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde399ce-bd66-4d4c-b46a-964d2d3fba13 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:38.665170 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde399ce-bd66-4d4c-b46a-964d2d3fba13 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:38.665541 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde399ce-bd66-4d4c-b46a-964d2d3fba13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:38.683313 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-dde399ce-bd66-4d4c-b46a-964d2d3fba13 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:38.684673 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-dde399ce-bd66-4d4c-b46a-964d2d3fba13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:38.703634 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:38.745224 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:38.760359 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 269/546] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:38.760359 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde399ce-bd66-4d4c-b46a-964d2d3fba13 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:38.764907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a1fc0-85b9-4db7-92af-5c5c35c26bb3 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:38.833278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b199ce-503e-4919-a2ed-777f41df2128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:38.834585 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b199ce-503e-4919-a2ed-777f41df2128 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:38.835086 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b199ce-503e-4919-a2ed-777f41df2128 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:38.836249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b199ce-503e-4919-a2ed-777f41df2128 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:38.837640 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b199ce-503e-4919-a2ed-777f41df2128 None admin] RBAC: Adding query filter params (role.id=5f27c72acb19431fa3223c962bc42938, scope.project.id=155180ba63c446ca889a6eb3f94cc31a, user.id=332fbbefba114da4a01fdd496fff1e31) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:38.838149 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b199ce-503e-4919-a2ed-777f41df2128 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:38.841047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b199ce-503e-4919-a2ed-777f41df2128 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:38.848611 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 278/547] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:48:38 2022] GET /identity/v3/role_assignments?user.id=332fbbefba114da4a01fdd496fff1e31&scope.project.id=155180ba63c446ca889a6eb3f94cc31a&role.id=5f27c72acb19431fa3223c962bc42938 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:38.849773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b199ce-503e-4919-a2ed-777f41df2128 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:38.873487 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde399ce-bd66-4d4c-b46a-964d2d3fba13 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:38.910237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02265a44-479a-40bd-a197-77a078a5e3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:38.911623 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02265a44-479a-40bd-a197-77a078a5e3e4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:38.911970 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02265a44-479a-40bd-a197-77a078a5e3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:38.912340 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02265a44-479a-40bd-a197-77a078a5e3e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:38.924825 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02265a44-479a-40bd-a197-77a078a5e3e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:38.926550 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02265a44-479a-40bd-a197-77a078a5e3e4 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:38.948386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-02265a44-479a-40bd-a197-77a078a5e3e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:38.948386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:38.948386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:38.948386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:38.948386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:38.948386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:38.948386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:38.948386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:38.948386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:38.948386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:38.948386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:38.948386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:38.948386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:38.948386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:38.948386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:38.949503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:38.949503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:38.949503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:38.949503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:38.949503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:38.949503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:38.949503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:38.949503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:38.949503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:38.949503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:38.949503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:38.949503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:38.949503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:38.949503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:38.949503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:38.949503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:38.949503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:38.951050 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:38.951050 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:38.951050 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:38.951050 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:38.951288 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 270/548] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:38.955937 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b4715-8c5f-4084-bbde-bc512a6251d9 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:38.955937 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b4715-8c5f-4084-bbde-bc512a6251d9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:38.955937 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044b4715-8c5f-4084-bbde-bc512a6251d9 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:38.955937 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 271/549] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:38.956876 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b4715-8c5f-4084-bbde-bc512a6251d9 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:38.974917 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:38.987233 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b199ce-503e-4919-a2ed-777f41df2128 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:39.030701 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27459d-0fd5-463b-832a-d67f64183c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:39.031846 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27459d-0fd5-463b-832a-d67f64183c98 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:39.032627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27459d-0fd5-463b-832a-d67f64183c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:39.032999 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27459d-0fd5-463b-832a-d67f64183c98 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:39.048382 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d27459d-0fd5-463b-832a-d67f64183c98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:39.056588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-4d27459d-0fd5-463b-832a-d67f64183c98 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:39.056588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:39.056588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:39.056588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:39.056588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:39.056588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:39.056588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:39.056588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:39.056588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:39.056588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:39.056588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:39.056588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:39.056588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:39.056588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:39.056588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:39.056588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:39.057502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:39.057502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:39.057502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:39.057502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:39.057502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:39.057502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:39.057502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:39.057502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:39.057502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:39.057502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:39.057502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:39.057502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:39.057502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:39.057502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:39.057502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:39.057502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:39.058315 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:39.058315 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:39.058315 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:39.058315 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:39.058315 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:39.058315 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:39.058315 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:39.058315 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:39.058315 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:39.058315 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:39.058315 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:39.058315 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:39.058315 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 279/550] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jun 23 19:48:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:39.059737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f03fc-2381-47c6-9f55-713fa79cbb23 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:39.060032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f03fc-2381-47c6-9f55-713fa79cbb23 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:39.060327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f03fc-2381-47c6-9f55-713fa79cbb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:39.077027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f80f03fc-2381-47c6-9f55-713fa79cbb23 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:39.078200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f80f03fc-2381-47c6-9f55-713fa79cbb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:39.085493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044b4715-8c5f-4084-bbde-bc512a6251d9 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:39.090703 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:39.146433 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5bd80-4175-41e0-a2b1-533e1ef851ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:39.147651 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5bd80-4175-41e0-a2b1-533e1ef851ff None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:39.148131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5bd80-4175-41e0-a2b1-533e1ef851ff None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:39.148717 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e5bd80-4175-41e0-a2b1-533e1ef851ff None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:39.149904 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e5bd80-4175-41e0-a2b1-533e1ef851ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:39.150414 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e5bd80-4175-41e0-a2b1-533e1ef851ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:39.152536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e5bd80-4175-41e0-a2b1-533e1ef851ff None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:39.168560 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 280/551] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:39.169618 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f03fc-2381-47c6-9f55-713fa79cbb23 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:39.175710 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 272/552] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:39.178040 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5bd80-4175-41e0-a2b1-533e1ef851ff None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:39.271336 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:39.310595 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e5bd80-4175-41e0-a2b1-533e1ef851ff None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:39.322253 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f03fc-2381-47c6-9f55-713fa79cbb23 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:39.350127 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b946d7-d312-4174-9db7-6d4e3138d285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:39.350678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b946d7-d312-4174-9db7-6d4e3138d285 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:39.350968 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b946d7-d312-4174-9db7-6d4e3138d285 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:39.351246 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b946d7-d312-4174-9db7-6d4e3138d285 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:39.362264 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b946d7-d312-4174-9db7-6d4e3138d285 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:39.364008 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b946d7-d312-4174-9db7-6d4e3138d285 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:39.367198 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358287e-8781-4e58-a271-2e9674c9dfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:39.368686 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358287e-8781-4e58-a271-2e9674c9dfaf None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:39.369389 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358287e-8781-4e58-a271-2e9674c9dfaf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:39.370125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358287e-8781-4e58-a271-2e9674c9dfaf None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:39.371747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358287e-8781-4e58-a271-2e9674c9dfaf None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:39.372392 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358287e-8781-4e58-a271-2e9674c9dfaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:39.374002 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358287e-8781-4e58-a271-2e9674c9dfaf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:39.380051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-98b946d7-d312-4174-9db7-6d4e3138d285 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 19:48:39.380051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:39.380051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:39.380051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:39.380051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:39.380051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:39.380051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:39.380051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:39.380051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:39.380051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:39.380051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:39.380051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:39.380051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:48:39.380051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:48:39.380051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:48:39.380051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:48:39.390329 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:39.390329 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:39.390329 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:48:39.390329 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:39.390329 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:48:39.390329 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:39.390329 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:39.390329 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:39.390329 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:39.390329 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:39.390329 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:39.390329 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:39.390329 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:39.390329 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:39.390329 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:39.390329 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:39.390329 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:39.390329 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:39.391976 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:39.391976 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:39.391976 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:48:39.391976 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:48:39.391976 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:48:39.391976 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:48:39.391976 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:48:39.391976 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:48:39.391976 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 19:48:39.391976 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:39.391976 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 273/553] 10.222.0.44 () {56 vars in 1093 bytes} [Thu Jun 23 19:48:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:39.391976 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b946d7-d312-4174-9db7-6d4e3138d285 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:39.392906 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 281/554] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jun 23 19:48:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:39.394918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358287e-8781-4e58-a271-2e9674c9dfaf None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:39.414004 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:39.524707 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b946d7-d312-4174-9db7-6d4e3138d285 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:39.531865 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358287e-8781-4e58-a271-2e9674c9dfaf None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:39.557661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:39.569029 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b3ed1a-2915-49a6-a763-433be4bf4208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:39.570069 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3ed1a-2915-49a6-a763-433be4bf4208 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:39.570501 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3ed1a-2915-49a6-a763-433be4bf4208 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:39.570614 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f6502-0130-479d-91d5-38ecef7220e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:39.571045 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3ed1a-2915-49a6-a763-433be4bf4208 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:39.571288 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f6502-0130-479d-91d5-38ecef7220e3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:39.571579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f6502-0130-479d-91d5-38ecef7220e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:39.571855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f6502-0130-479d-91d5-38ecef7220e3 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:39.572726 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f6502-0130-479d-91d5-38ecef7220e3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:39.573035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f6502-0130-479d-91d5-38ecef7220e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:39.574463 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f6502-0130-479d-91d5-38ecef7220e3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:39.586148 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b3ed1a-2915-49a6-a763-433be4bf4208 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:39.593889 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 282/555] 10.222.0.44 () {56 vars in 1133 bytes} [Thu Jun 23 19:48:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:39.594717 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f6502-0130-479d-91d5-38ecef7220e3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:39.595379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-41b3ed1a-2915-49a6-a763-433be4bf4208 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:39.595379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:39.595379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:39.595379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:39.595379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:39.595379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:39.595379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:39.595379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:39.595379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:39.595379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:39.595379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:39.595379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:39.595379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:39.595379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:39.595379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:39.595379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:39.596466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:39.596466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:39.596466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:39.596466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:39.596466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:39.596466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:39.596466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:39.596466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:39.596466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:39.596466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:39.596466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:39.596466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:39.596466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:39.596466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:39.596466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:39.596466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:39.597570 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:39.597570 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:39.597570 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:39.597570 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:39.597570 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:39.597570 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:39.597570 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:39.597570 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:39.597570 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:39.597570 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:39.597570 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:39.597570 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:39.606598 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 274/556] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jun 23 19:48:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:39.607678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b3ed1a-2915-49a6-a763-433be4bf4208 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:39.675185 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:39.714782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f6502-0130-479d-91d5-38ecef7220e3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:39.738473 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b3ed1a-2915-49a6-a763-433be4bf4208 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:39.765309 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5214de1-26cc-4201-ba2c-49af7298975d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:39.766118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5214de1-26cc-4201-ba2c-49af7298975d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:39.766401 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5214de1-26cc-4201-ba2c-49af7298975d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:39.766671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5214de1-26cc-4201-ba2c-49af7298975d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:39.782917 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5214de1-26cc-4201-ba2c-49af7298975d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:39.783504 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bfe832-906f-4929-ae84-f408deacf685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:39.784425 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5214de1-26cc-4201-ba2c-49af7298975d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:39.784732 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfe832-906f-4929-ae84-f408deacf685 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:39.785362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfe832-906f-4929-ae84-f408deacf685 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:39.785854 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfe832-906f-4929-ae84-f408deacf685 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:39.799779 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bfe832-906f-4929-ae84-f408deacf685 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:39.802738 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bfe832-906f-4929-ae84-f408deacf685 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:39.808653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-d5214de1-26cc-4201-ba2c-49af7298975d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:39.808653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:39.808653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:39.808653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:39.808653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:39.808653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:39.808653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:39.808653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:39.808653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:39.808653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:39.808653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:39.808653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:39.808653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:39.808653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:39.808653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:39.810029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:39.810029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:39.810029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:39.810029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:39.810029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:39.810029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:39.810029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:39.810029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:39.810029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:39.810029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:39.810029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:39.810029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:39.810029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:39.810029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:39.810029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:39.810029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:39.810029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:39.810956 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:39.810956 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:39.810956 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:39.810956 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:39.810956 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 283/557] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:39.810956 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5214de1-26cc-4201-ba2c-49af7298975d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:39.833997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-96bfe832-906f-4929-ae84-f408deacf685 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:39.833997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:39.833997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:39.833997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:39.833997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:39.833997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:39.833997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:39.833997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:39.833997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:39.833997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:39.833997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:39.833997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:39.833997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:39.833997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:39.833997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:39.834954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:39.834954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:39.834954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:39.834954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:39.834954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:39.834954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:39.834954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:39.834954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:39.834954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:39.834954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:39.834954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:39.834954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:39.834954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:39.834954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:39.834954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:39.834954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:39.834954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:39.835881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:39.835881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:39.835881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:39.835881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:39.838159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 275/558] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:39.839483 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bfe832-906f-4929-ae84-f408deacf685 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:39.897911 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:39.950573 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5214de1-26cc-4201-ba2c-49af7298975d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:39.976670 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:39.980433 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bfe832-906f-4929-ae84-f408deacf685 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:39.992913 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146cee12-28bc-454d-86c8-2bb875d0f56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:39.994046 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146cee12-28bc-454d-86c8-2bb875d0f56e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:39.994517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146cee12-28bc-454d-86c8-2bb875d0f56e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:39.995360 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146cee12-28bc-454d-86c8-2bb875d0f56e None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:39.996578 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146cee12-28bc-454d-86c8-2bb875d0f56e None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:39.997093 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146cee12-28bc-454d-86c8-2bb875d0f56e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:39.999441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146cee12-28bc-454d-86c8-2bb875d0f56e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:40.016824 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 284/559] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jun 23 19:48:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:40.017571 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146cee12-28bc-454d-86c8-2bb875d0f56e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:40.022076 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d136a-f3d6-4010-bd79-a24d1c62c1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:40.023141 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d136a-f3d6-4010-bd79-a24d1c62c1a9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:40.023141 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d136a-f3d6-4010-bd79-a24d1c62c1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:40.023344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d136a-f3d6-4010-bd79-a24d1c62c1a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:40.024381 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d136a-f3d6-4010-bd79-a24d1c62c1a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:40.024689 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d136a-f3d6-4010-bd79-a24d1c62c1a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:40.026090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d136a-f3d6-4010-bd79-a24d1c62c1a9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:40.040123 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 276/560] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:40.041572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d136a-f3d6-4010-bd79-a24d1c62c1a9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:40.137049 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146cee12-28bc-454d-86c8-2bb875d0f56e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:40.159460 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:40.165015 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d136a-f3d6-4010-bd79-a24d1c62c1a9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:40.177908 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c87a4d-49e6-4298-a009-918581cb636d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:40.184141 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c87a4d-49e6-4298-a009-918581cb636d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:40.184141 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c87a4d-49e6-4298-a009-918581cb636d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:40.184141 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c87a4d-49e6-4298-a009-918581cb636d None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:40.184141 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c87a4d-49e6-4298-a009-918581cb636d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:40.184141 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c87a4d-49e6-4298-a009-918581cb636d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:40.184141 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c87a4d-49e6-4298-a009-918581cb636d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:40.194841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 285/561] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:40.195937 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c87a4d-49e6-4298-a009-918581cb636d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:40.213953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ae523-ceeb-4808-80d7-21ef6a3daac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:40.215083 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ae523-ceeb-4808-80d7-21ef6a3daac4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:40.215601 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ae523-ceeb-4808-80d7-21ef6a3daac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:40.216076 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ae523-ceeb-4808-80d7-21ef6a3daac4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:40.228644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ae523-ceeb-4808-80d7-21ef6a3daac4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:40.230429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ae523-ceeb-4808-80d7-21ef6a3daac4 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:40.252646 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:40.260788 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-105ae523-ceeb-4808-80d7-21ef6a3daac4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:40.260788 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:40.260788 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:40.260788 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:40.260788 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:40.260788 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:40.260788 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:40.260788 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:40.260788 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:40.260788 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:40.260788 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:40.260788 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:40.260788 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:40.260788 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:40.260788 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:40.268547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:40.268547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:40.268547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:40.268547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:40.268547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:40.268547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:40.268547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:40.268547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:40.268547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:40.268547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:40.268547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:40.268547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:40.268547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:40.268547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:40.268547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:40.268547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:40.268547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:40.269436 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:40.269436 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:40.269436 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:40.269436 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:40.269436 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 277/562] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:40.269436 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ae523-ceeb-4808-80d7-21ef6a3daac4 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:40.324507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c87a4d-49e6-4298-a009-918581cb636d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:40.359340 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ae78e-0015-4895-a9c7-71aa7db19c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:40.360416 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ae78e-0015-4895-a9c7-71aa7db19c03 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:40.360862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ae78e-0015-4895-a9c7-71aa7db19c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:40.361329 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ae78e-0015-4895-a9c7-71aa7db19c03 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:40.369885 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ae78e-0015-4895-a9c7-71aa7db19c03 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:40.371966 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724ae78e-0015-4895-a9c7-71aa7db19c03 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:40.385890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ae523-ceeb-4808-80d7-21ef6a3daac4 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:40.387741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-724ae78e-0015-4895-a9c7-71aa7db19c03 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 19:48:40.387741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:40.387741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:40.387741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:40.387741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:40.387741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:40.387741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:40.387741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:40.387741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:40.387741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:40.387741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:40.387741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:40.387741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:48:40.387741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:48:40.387741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:48:40.387741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:48:40.394200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:40.394200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:40.394200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:48:40.394200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:40.394200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:48:40.394200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:40.394200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:40.394200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:40.394200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:40.394200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:40.394200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:40.394200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:40.394200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:40.394200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:40.394200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:40.394200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:40.394200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:40.394200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:40.395317 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:40.395317 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:40.395317 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:48:40.395317 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:48:40.395317 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:48:40.395317 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:48:40.395317 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:48:40.395317 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:48:40.395317 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 19:48:40.395317 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:40.395317 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 286/563] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:48:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:40.395317 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ae78e-0015-4895-a9c7-71aa7db19c03 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:40.422189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e5112-6d6c-4a12-b613-561774cda8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:40.427516 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e5112-6d6c-4a12-b613-561774cda8a1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:40.427516 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e5112-6d6c-4a12-b613-561774cda8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:40.427516 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e5112-6d6c-4a12-b613-561774cda8a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:40.430201 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e5112-6d6c-4a12-b613-561774cda8a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:40.431663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e5112-6d6c-4a12-b613-561774cda8a1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:40.445908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-ae0e5112-6d6c-4a12-b613-561774cda8a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:40.445908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:40.445908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:40.445908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:40.445908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:40.445908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:40.445908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:40.445908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:40.445908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:40.445908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:40.445908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:40.445908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:40.445908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:48:40.445908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:48:40.445908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:48:40.446918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:48:40.446918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:40.446918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:40.446918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:40.446918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:40.446918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:40.446918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:40.446918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:40.446918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:40.446918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:40.446918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:40.446918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:40.446918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:40.446918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:40.446918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:40.446918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:40.446918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:40.447828 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:48:40.447828 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:48:40.447828 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:48:40.447828 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:48:40.447828 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:48:40.447828 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:48:40.447828 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:40.447828 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:40.447828 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 278/564] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:48:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:40.448643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343a409-9316-4cd7-8a12-4f8062fa9df7 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:40.448933 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343a409-9316-4cd7-8a12-4f8062fa9df7 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:40.449211 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343a409-9316-4cd7-8a12-4f8062fa9df7 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:40.450576 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 279/565] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:40.454861 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343a409-9316-4cd7-8a12-4f8062fa9df7 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:40.467344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:40.524444 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:40.527510 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724ae78e-0015-4895-a9c7-71aa7db19c03 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:40.562444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343a409-9316-4cd7-8a12-4f8062fa9df7 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:40.564952 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1868f-38e7-424e-a8e6-8b9b4d9fc75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:40.565608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1868f-38e7-424e-a8e6-8b9b4d9fc75d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:40.565889 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1868f-38e7-424e-a8e6-8b9b4d9fc75d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:40.566164 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1868f-38e7-424e-a8e6-8b9b4d9fc75d None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:40.566874 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae1868f-38e7-424e-a8e6-8b9b4d9fc75d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:40.567179 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae1868f-38e7-424e-a8e6-8b9b4d9fc75d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:40.568635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae1868f-38e7-424e-a8e6-8b9b4d9fc75d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:40.584433 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 287/566] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:40.584433 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1868f-38e7-424e-a8e6-8b9b4d9fc75d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:40.603291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27430b2-343f-40a9-907b-bac6656d1f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:40.603963 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27430b2-343f-40a9-907b-bac6656d1f4a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:40.604247 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27430b2-343f-40a9-907b-bac6656d1f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:40.604558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27430b2-343f-40a9-907b-bac6656d1f4a None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:40.605407 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27430b2-343f-40a9-907b-bac6656d1f4a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:40.605707 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27430b2-343f-40a9-907b-bac6656d1f4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:40.607142 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27430b2-343f-40a9-907b-bac6656d1f4a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:40.625378 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 280/567] 10.222.0.44 () {56 vars in 1135 bytes} [Thu Jun 23 19:48:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:40.626435 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ec1df-6354-4972-b6af-1343988023b2 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:40.626727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ec1df-6354-4972-b6af-1343988023b2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:40.627006 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452ec1df-6354-4972-b6af-1343988023b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:40.643222 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-452ec1df-6354-4972-b6af-1343988023b2 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:40.644456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-452ec1df-6354-4972-b6af-1343988023b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:40.695211 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1868f-38e7-424e-a8e6-8b9b4d9fc75d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:40.707823 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:40.713752 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 281/568] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:40.714552 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ec1df-6354-4972-b6af-1343988023b2 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:40.734321 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d88c1-f725-43ad-a932-7601eaf5f2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:40.735267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d88c1-f725-43ad-a932-7601eaf5f2d6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:40.735724 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d88c1-f725-43ad-a932-7601eaf5f2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:40.736177 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d88c1-f725-43ad-a932-7601eaf5f2d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:40.737163 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d88c1-f725-43ad-a932-7601eaf5f2d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:40.737646 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d88c1-f725-43ad-a932-7601eaf5f2d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:40.739340 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d88c1-f725-43ad-a932-7601eaf5f2d6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:40.752154 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 288/569] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:48:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:40.753293 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d88c1-f725-43ad-a932-7601eaf5f2d6 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:40.778634 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:40.838321 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452ec1df-6354-4972-b6af-1343988023b2 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:40.865651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d88c1-f725-43ad-a932-7601eaf5f2d6 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:40.875217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc00e4-77f4-48e2-9eb3-c062d9ed3825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:40.876373 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc00e4-77f4-48e2-9eb3-c062d9ed3825 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:40.876827 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc00e4-77f4-48e2-9eb3-c062d9ed3825 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:40.877264 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc00e4-77f4-48e2-9eb3-c062d9ed3825 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:40.885938 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc00e4-77f4-48e2-9eb3-c062d9ed3825 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:40.887465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc00e4-77f4-48e2-9eb3-c062d9ed3825 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:40.899221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a1a52-7618-45cd-b86e-a9422439b941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:40.899865 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a1a52-7618-45cd-b86e-a9422439b941 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:40.900152 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a1a52-7618-45cd-b86e-a9422439b941 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:40.900460 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a1a52-7618-45cd-b86e-a9422439b941 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:40.902427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-edbc00e4-77f4-48e2-9eb3-c062d9ed3825 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 19:48:40.902427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:40.902427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:40.902427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:40.902427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:40.902427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:40.902427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:40.902427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:40.902427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:40.902427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:40.902427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:40.902427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:40.902427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:48:40.902427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:48:40.902427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:48:40.902427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:48:40.903313 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:40.903313 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:40.903313 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:48:40.903313 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:40.903313 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:48:40.903313 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:40.903313 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:40.903313 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:40.903313 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:40.903313 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:40.903313 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:40.903313 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:40.903313 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:40.903313 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:40.903313 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:40.903313 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:40.903313 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:40.903313 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:40.904700 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:40.904700 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:40.904700 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:48:40.904700 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:48:40.904700 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:48:40.904700 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:48:40.904700 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:48:40.904700 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:48:40.904700 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 19:48:40.904700 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:40.904700 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 282/570] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:40.904700 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc00e4-77f4-48e2-9eb3-c062d9ed3825 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:40.964701 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:40.976444 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a1a52-7618-45cd-b86e-a9422439b941 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:40.977925 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a1a52-7618-45cd-b86e-a9422439b941 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:41.001084 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:41.011656 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc00e4-77f4-48e2-9eb3-c062d9ed3825 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:41.048384 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7da2ce-e452-428e-95ba-3e5453a5c2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:41.049181 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7da2ce-e452-428e-95ba-3e5453a5c2d9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:41.049466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7da2ce-e452-428e-95ba-3e5453a5c2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:41.049744 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7da2ce-e452-428e-95ba-3e5453a5c2d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:41.060639 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7da2ce-e452-428e-95ba-3e5453a5c2d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:41.068604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-4c7da2ce-e452-428e-95ba-3e5453a5c2d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:41.068604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:41.068604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:41.068604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:41.068604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:41.068604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:41.068604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:41.068604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:41.068604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:41.068604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:41.068604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:41.068604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:41.068604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:41.068604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:41.068604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:41.068604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:41.069536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:41.069536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:41.069536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:41.069536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:41.069536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:41.069536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:41.069536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:41.069536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:41.069536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:41.069536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:41.069536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:41.069536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:41.069536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:41.069536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:41.069536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:41.069536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:41.070382 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:41.070382 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:41.070382 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:41.070382 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:41.070382 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:41.070382 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:41.070382 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:41.070382 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:41.070382 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:41.070382 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:41.070382 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:41.070382 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:41.070382 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 283/571] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jun 23 19:48:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:41.071098 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7da2ce-e452-428e-95ba-3e5453a5c2d9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:41.092462 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 289/572] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:48:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:48:41.093494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a1a52-7618-45cd-b86e-a9422439b941 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:41.180975 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7da2ce-e452-428e-95ba-3e5453a5c2d9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:41.197910 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:41.200713 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a1a52-7618-45cd-b86e-a9422439b941 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:41.214991 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f124313-e4df-4288-bdf6-1bb8a2181553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:41.215875 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f124313-e4df-4288-bdf6-1bb8a2181553 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:41.216155 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f124313-e4df-4288-bdf6-1bb8a2181553 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:41.216461 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f124313-e4df-4288-bdf6-1bb8a2181553 None admin] PATH_INFO: `/v3/projects/155180ba63c446ca889a6eb3f94cc31a/users/a63d1d902c024c97a38f7d92f73814d3/roles/5f27c72acb19431fa3223c962bc42938` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:41.220161 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:41.231037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f124313-e4df-4288-bdf6-1bb8a2181553 None admin] RBAC: Authorizing `identity:create_grant(project_id=155180ba63c446ca889a6eb3f94cc31a, user_id=a63d1d902c024c97a38f7d92f73814d3, role_id=5f27c72acb19431fa3223c962bc42938)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:41.232742 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f124313-e4df-4288-bdf6-1bb8a2181553 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:41.235335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab2682-d511-4120-a34c-32ba1200d02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:41.238361 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab2682-d511-4120-a34c-32ba1200d02d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:41.238361 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab2682-d511-4120-a34c-32ba1200d02d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:41.238361 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cab2682-d511-4120-a34c-32ba1200d02d None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:41.238361 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab2682-d511-4120-a34c-32ba1200d02d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:41.238361 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab2682-d511-4120-a34c-32ba1200d02d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:41.239554 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cab2682-d511-4120-a34c-32ba1200d02d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:41.256982 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 290/573] 10.222.0.44 () {56 vars in 1137 bytes} [Thu Jun 23 19:48:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:41.257695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab2682-d511-4120-a34c-32ba1200d02d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:41.318966 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 284/574] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jun 23 19:48:41 2022] PUT /identity/v3/projects/155180ba63c446ca889a6eb3f94cc31a/users/a63d1d902c024c97a38f7d92f73814d3/roles/5f27c72acb19431fa3223c962bc42938 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:48:41.319922 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f124313-e4df-4288-bdf6-1bb8a2181553 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:41.381092 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cab2682-d511-4120-a34c-32ba1200d02d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:41.420829 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45223400-5185-4232-b91d-196652cedecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:41.421513 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45223400-5185-4232-b91d-196652cedecd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:41.421850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45223400-5185-4232-b91d-196652cedecd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:41.422134 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45223400-5185-4232-b91d-196652cedecd None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:41.422895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45223400-5185-4232-b91d-196652cedecd None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:41.423206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45223400-5185-4232-b91d-196652cedecd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:41.424910 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45223400-5185-4232-b91d-196652cedecd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:41.430532 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f124313-e4df-4288-bdf6-1bb8a2181553 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:41.434604 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 291/575] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jun 23 19:48:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:41.435779 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f675f-bbd7-4cd7-a78e-60d3eb7535ec None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:41.436073 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f675f-bbd7-4cd7-a78e-60d3eb7535ec None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:41.436384 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f675f-bbd7-4cd7-a78e-60d3eb7535ec None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:41.441921 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 292/576] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:41.441921 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f675f-bbd7-4cd7-a78e-60d3eb7535ec None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:41.480718 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ddc35c-dc37-4858-98c8-280cce603f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:41.481520 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ddc35c-dc37-4858-98c8-280cce603f7a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:41.481840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ddc35c-dc37-4858-98c8-280cce603f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:41.482081 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ddc35c-dc37-4858-98c8-280cce603f7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:41.493647 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ddc35c-dc37-4858-98c8-280cce603f7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:41.495444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ddc35c-dc37-4858-98c8-280cce603f7a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:41.495555 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:41.514662 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-96ddc35c-dc37-4858-98c8-280cce603f7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:41.514662 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:41.514662 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:41.514662 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:41.514662 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:41.514662 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:41.514662 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:41.514662 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:41.514662 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:41.514662 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:41.514662 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:41.514662 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:41.514662 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:41.514662 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:41.514662 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:41.515814 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:41.515814 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:41.515814 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:41.515814 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:41.515814 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:41.515814 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:41.515814 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:41.515814 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:41.515814 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:41.515814 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:41.515814 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:41.515814 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:41.515814 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:41.515814 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:41.515814 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:41.515814 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:41.515814 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:41.516744 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:41.516744 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:41.516744 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:41.516744 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:41.516744 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 285/577] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:41.517030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ddc35c-dc37-4858-98c8-280cce603f7a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:41.543931 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:41.564861 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f675f-bbd7-4cd7-a78e-60d3eb7535ec None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:41.601782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac5c56-37fd-43cc-960f-2df928b07327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:41.602576 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac5c56-37fd-43cc-960f-2df928b07327 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:41.603063 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac5c56-37fd-43cc-960f-2df928b07327 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:41.603319 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac5c56-37fd-43cc-960f-2df928b07327 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:41.613360 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac5c56-37fd-43cc-960f-2df928b07327 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:41.615329 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac5c56-37fd-43cc-960f-2df928b07327 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:41.627137 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ddc35c-dc37-4858-98c8-280cce603f7a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:41.633633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-daac5c56-37fd-43cc-960f-2df928b07327 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:41.633633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:41.633633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:41.633633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:41.633633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:41.633633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:41.633633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:41.633633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:41.633633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:41.633633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:41.633633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:41.633633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:41.633633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:41.633633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:41.633633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:41.634648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:41.634648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:41.634648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:41.634648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:41.634648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:41.634648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:41.634648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:41.634648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:41.634648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:41.634648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:41.634648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:41.634648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:41.634648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:41.634648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:41.634648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:41.634648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:41.634648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:41.635725 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:41.635725 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:41.635725 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:41.635725 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:41.637560 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 293/578] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:41.638911 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e01611-3510-481e-9faf-0caa87a8706f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:41.639194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e01611-3510-481e-9faf-0caa87a8706f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:41.639466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e01611-3510-481e-9faf-0caa87a8706f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:41.656100 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-28e01611-3510-481e-9faf-0caa87a8706f None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:41.657317 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-28e01611-3510-481e-9faf-0caa87a8706f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:41.668254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b36f4-cfc2-4e85-b1f0-2e1fc3a67777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:41.669258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b36f4-cfc2-4e85-b1f0-2e1fc3a67777 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:41.669538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b36f4-cfc2-4e85-b1f0-2e1fc3a67777 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:41.669809 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b36f4-cfc2-4e85-b1f0-2e1fc3a67777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:41.680984 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b36f4-cfc2-4e85-b1f0-2e1fc3a67777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:41.682444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b36f4-cfc2-4e85-b1f0-2e1fc3a67777 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:41.701458 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-b87b36f4-cfc2-4e85-b1f0-2e1fc3a67777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:41.701458 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:41.701458 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:41.701458 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:41.701458 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:41.701458 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:41.701458 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:41.701458 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:41.701458 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:41.701458 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:41.701458 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:41.701458 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:41.701458 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:41.701458 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:41.701458 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:41.702346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:41.702346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:41.702346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:41.702346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:41.702346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:41.702346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:41.702346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:41.702346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:41.702346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:41.702346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:41.702346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:41.702346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:41.702346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:41.702346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:41.702346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:41.702346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:41.702346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:41.703266 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:41.703266 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:41.703266 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:41.703266 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:41.703266 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 286/579] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:41.703545 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b36f4-cfc2-4e85-b1f0-2e1fc3a67777 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:41.744442 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 294/580] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:41.745538 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e01611-3510-481e-9faf-0caa87a8706f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:41.778539 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:41.779730 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:41.810908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b36f4-cfc2-4e85-b1f0-2e1fc3a67777 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:41.846111 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b31c6-6ebf-464a-a13f-fa15c1597fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:41.848099 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b31c6-6ebf-464a-a13f-fa15c1597fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:41.848818 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b31c6-6ebf-464a-a13f-fa15c1597fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:41.849345 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b31c6-6ebf-464a-a13f-fa15c1597fe9 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:41.850748 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b31c6-6ebf-464a-a13f-fa15c1597fe9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:41.851282 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b31c6-6ebf-464a-a13f-fa15c1597fe9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:41.853242 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b31c6-6ebf-464a-a13f-fa15c1597fe9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:41.866702 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e01611-3510-481e-9faf-0caa87a8706f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:41.868457 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 287/581] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:41.869341 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b31c6-6ebf-464a-a13f-fa15c1597fe9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:41.906683 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73588685-6096-4b1b-8868-4e95a5743d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:41.907339 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73588685-6096-4b1b-8868-4e95a5743d93 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:41.907640 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73588685-6096-4b1b-8868-4e95a5743d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:41.907901 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73588685-6096-4b1b-8868-4e95a5743d93 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:41.908642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73588685-6096-4b1b-8868-4e95a5743d93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:41.908950 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73588685-6096-4b1b-8868-4e95a5743d93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:41.910318 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73588685-6096-4b1b-8868-4e95a5743d93 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:41.921440 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 295/582] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:41.922247 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73588685-6096-4b1b-8868-4e95a5743d93 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:41.990534 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b31c6-6ebf-464a-a13f-fa15c1597fe9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:42.011391 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:42.022250 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:42.027346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6531ef3-6606-4b41-bd7a-4c64e26d7e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:42.028189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6531ef3-6606-4b41-bd7a-4c64e26d7e8d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:42.029304 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6531ef3-6606-4b41-bd7a-4c64e26d7e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:42.029304 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6531ef3-6606-4b41-bd7a-4c64e26d7e8d None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:42.030142 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6531ef3-6606-4b41-bd7a-4c64e26d7e8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:42.030480 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73588685-6096-4b1b-8868-4e95a5743d93 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:42.030846 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6531ef3-6606-4b41-bd7a-4c64e26d7e8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:42.032451 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6531ef3-6606-4b41-bd7a-4c64e26d7e8d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:42.048832 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 288/583] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:42.049969 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6531ef3-6606-4b41-bd7a-4c64e26d7e8d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:42.065165 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce33a1d-8e46-4aab-b283-6f49b7b52345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:42.065957 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce33a1d-8e46-4aab-b283-6f49b7b52345 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:42.066245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce33a1d-8e46-4aab-b283-6f49b7b52345 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:42.066519 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce33a1d-8e46-4aab-b283-6f49b7b52345 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:42.077818 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce33a1d-8e46-4aab-b283-6f49b7b52345 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:42.085516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-1ce33a1d-8e46-4aab-b283-6f49b7b52345 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:42.085516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:42.085516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:42.085516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:42.085516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:42.085516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:42.085516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:42.085516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:42.085516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:42.085516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:42.085516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:42.085516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:42.085516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:42.085516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:42.085516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:42.085516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:42.086458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:42.086458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:42.086458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:42.086458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:42.086458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:42.086458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:42.086458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:42.086458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:42.086458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:42.086458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:42.086458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:42.086458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:42.086458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:42.086458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:42.086458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:42.086458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:42.087279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:42.087279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:42.087279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:42.087279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:42.087279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:42.087279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:42.087279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:42.087279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:42.087279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:42.087279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:42.087279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:42.087279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:42.087279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 296/584] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jun 23 19:48:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:42.087958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce33a1d-8e46-4aab-b283-6f49b7b52345 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:42.168907 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6531ef3-6606-4b41-bd7a-4c64e26d7e8d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:42.203096 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce33a1d-8e46-4aab-b283-6f49b7b52345 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:42.204470 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fdeab5-2e21-4dee-ae8b-a02dce1ce1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:42.205246 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdeab5-2e21-4dee-ae8b-a02dce1ce1cf None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:42.205523 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdeab5-2e21-4dee-ae8b-a02dce1ce1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:42.205792 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdeab5-2e21-4dee-ae8b-a02dce1ce1cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:42.212518 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fdeab5-2e21-4dee-ae8b-a02dce1ce1cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:42.213959 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fdeab5-2e21-4dee-ae8b-a02dce1ce1cf None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:42.228742 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-72fdeab5-2e21-4dee-ae8b-a02dce1ce1cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:42.228742 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:42.228742 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:42.228742 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:42.228742 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:42.228742 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:42.228742 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:42.228742 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:42.228742 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:42.228742 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:42.228742 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:42.228742 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:42.228742 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:48:42.228742 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:48:42.228742 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:48:42.229648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:48:42.229648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:42.229648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:42.229648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:42.229648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:42.229648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:42.229648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:42.229648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:42.229648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:42.229648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:42.229648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:42.229648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:42.229648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:42.229648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:42.229648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:42.229648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:42.229648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:42.231016 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:48:42.231016 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:48:42.231016 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:48:42.231016 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:48:42.231016 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:48:42.231016 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:48:42.231016 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:42.231016 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:42.231016 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 289/585] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:48:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:42.231016 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fdeab5-2e21-4dee-ae8b-a02dce1ce1cf None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:42.248527 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84aa35-52bb-43ee-9419-7f1d4c331ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:42.249331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84aa35-52bb-43ee-9419-7f1d4c331ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:42.249617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84aa35-52bb-43ee-9419-7f1d4c331ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:42.249892 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84aa35-52bb-43ee-9419-7f1d4c331ea1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:42.257262 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84aa35-52bb-43ee-9419-7f1d4c331ea1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:42.258724 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84aa35-52bb-43ee-9419-7f1d4c331ea1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:42.272756 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-ca84aa35-52bb-43ee-9419-7f1d4c331ea1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:42.272756 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:42.272756 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:42.272756 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:42.272756 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:42.272756 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:42.272756 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:42.272756 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:42.272756 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:42.272756 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:42.272756 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:42.272756 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:42.272756 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:48:42.272756 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:48:42.272756 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:48:42.273627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:48:42.273627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:42.273627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:42.273627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:42.273627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:42.273627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:42.273627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:42.273627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:42.273627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:42.273627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:42.273627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:42.273627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:42.273627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:42.273627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:42.273627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:42.273627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:42.273627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:42.274492 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:48:42.274492 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:48:42.274492 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:48:42.274492 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:48:42.274492 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:48:42.274492 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:48:42.274492 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:42.274492 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:42.274492 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 297/586] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:48:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:42.274957 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84aa35-52bb-43ee-9419-7f1d4c331ea1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:42.280765 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:42.294512 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:42.347304 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fdeab5-2e21-4dee-ae8b-a02dce1ce1cf None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:42.382553 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84aa35-52bb-43ee-9419-7f1d4c331ea1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:42.382748 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6451e736-9567-40e7-a50b-7a4bb08cd64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:42.383516 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6451e736-9567-40e7-a50b-7a4bb08cd64d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:42.383829 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6451e736-9567-40e7-a50b-7a4bb08cd64d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:42.384190 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6451e736-9567-40e7-a50b-7a4bb08cd64d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:42.391906 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6451e736-9567-40e7-a50b-7a4bb08cd64d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:42.393415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6451e736-9567-40e7-a50b-7a4bb08cd64d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:42.408000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-6451e736-9567-40e7-a50b-7a4bb08cd64d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 19:48:42.408000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:42.408000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:42.408000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:42.408000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:42.408000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:42.408000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:42.408000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:42.408000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:42.408000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:42.408000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:42.408000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:42.408000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:48:42.408000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:48:42.408000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:48:42.408000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:48:42.408983 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:42.408983 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:42.408983 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:48:42.408983 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:42.408983 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:48:42.408983 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:42.408983 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:42.408983 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:42.408983 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:42.408983 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:42.408983 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:42.408983 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:42.408983 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:42.408983 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:42.408983 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:42.408983 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:42.408983 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:42.408983 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:42.409916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:42.409916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:42.409916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:48:42.409916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:48:42.409916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:48:42.409916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:48:42.409916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:48:42.409916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:48:42.409916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 19:48:42.409916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:42.409916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 290/587] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:42.410717 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6451e736-9567-40e7-a50b-7a4bb08cd64d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:42.426602 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40ad9c-332a-48c7-aa63-3325f16ebb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:42.427815 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40ad9c-332a-48c7-aa63-3325f16ebb97 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:42.428096 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40ad9c-332a-48c7-aa63-3325f16ebb97 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:42.428398 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40ad9c-332a-48c7-aa63-3325f16ebb97 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:42.429162 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e40ad9c-332a-48c7-aa63-3325f16ebb97 None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:42.429474 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e40ad9c-332a-48c7-aa63-3325f16ebb97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:42.430919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e40ad9c-332a-48c7-aa63-3325f16ebb97 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:42.444924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 298/588] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jun 23 19:48:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:42.445803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40ad9c-332a-48c7-aa63-3325f16ebb97 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:42.516113 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:42.554426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6451e736-9567-40e7-a50b-7a4bb08cd64d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:42.557193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40ad9c-332a-48c7-aa63-3325f16ebb97 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:42.567966 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:42.589901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f80d6-5b8f-489d-962f-4dfdb1ca32f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:42.590843 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f80d6-5b8f-489d-962f-4dfdb1ca32f7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:42.591327 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f80d6-5b8f-489d-962f-4dfdb1ca32f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:42.591802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48f80d6-5b8f-489d-962f-4dfdb1ca32f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:42.592913 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48f80d6-5b8f-489d-962f-4dfdb1ca32f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:42.593094 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e5433-bbd0-49c6-a911-7e63263eb231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:42.593611 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48f80d6-5b8f-489d-962f-4dfdb1ca32f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:42.593733 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e5433-bbd0-49c6-a911-7e63263eb231 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:42.593997 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e5433-bbd0-49c6-a911-7e63263eb231 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:42.594275 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e5433-bbd0-49c6-a911-7e63263eb231 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:42.595056 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e5433-bbd0-49c6-a911-7e63263eb231 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:42.595366 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e5433-bbd0-49c6-a911-7e63263eb231 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:42.596234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48f80d6-5b8f-489d-962f-4dfdb1ca32f7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:42.596920 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e5433-bbd0-49c6-a911-7e63263eb231 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:42.608915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 299/589] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:48:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:42.609952 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e5433-bbd0-49c6-a911-7e63263eb231 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:42.615239 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 291/590] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:48:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:42.616163 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f80d6-5b8f-489d-962f-4dfdb1ca32f7 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:42.714633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e5433-bbd0-49c6-a911-7e63263eb231 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:42.719464 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48f80d6-5b8f-489d-962f-4dfdb1ca32f7 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:42.740965 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:42.752509 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dcab77-eea2-4b64-bb5f-32930c0698f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:42.753272 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dcab77-eea2-4b64-bb5f-32930c0698f6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:42.753615 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dcab77-eea2-4b64-bb5f-32930c0698f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:42.753948 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dcab77-eea2-4b64-bb5f-32930c0698f6 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:42.767597 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526d8bc-c64b-4926-b6fc-ffc9faae4b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:42.767737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dcab77-eea2-4b64-bb5f-32930c0698f6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:42.767737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dcab77-eea2-4b64-bb5f-32930c0698f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:42.767913 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526d8bc-c64b-4926-b6fc-ffc9faae4b38 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:42.767913 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526d8bc-c64b-4926-b6fc-ffc9faae4b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:42.767913 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526d8bc-c64b-4926-b6fc-ffc9faae4b38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:42.768156 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dcab77-eea2-4b64-bb5f-32930c0698f6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:42.768240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526d8bc-c64b-4926-b6fc-ffc9faae4b38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:42.769823 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526d8bc-c64b-4926-b6fc-ffc9faae4b38 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:42.779676 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 300/591] 10.222.0.44 () {56 vars in 1137 bytes} [Thu Jun 23 19:48:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:42.779676 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dcab77-eea2-4b64-bb5f-32930c0698f6 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:42.791160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-4526d8bc-c64b-4926-b6fc-ffc9faae4b38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:42.791160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:42.791160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:42.791160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:42.791160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:42.791160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:42.791160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:42.791160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:42.791160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:42.791160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:42.791160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:42.791160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:42.791160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:42.791160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:42.791160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:42.792258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:42.792258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:42.792258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:42.792258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:42.792258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:42.792258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:42.792258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:42.792258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:42.792258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:42.792258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:42.792258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:42.792258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:42.792258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:42.792258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:42.792258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:42.792258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:42.792258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:42.793407 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:42.793407 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:42.793407 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:42.793407 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:42.799146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 292/592] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:42.799146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526d8bc-c64b-4926-b6fc-ffc9faae4b38 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:42.823964 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:42.884765 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dcab77-eea2-4b64-bb5f-32930c0698f6 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:42.898145 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526d8bc-c64b-4926-b6fc-ffc9faae4b38 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:42.920334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acddb4f9-d780-4320-9b7c-e5d5b1a1760e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:42.921233 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acddb4f9-d780-4320-9b7c-e5d5b1a1760e None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:42.921512 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acddb4f9-d780-4320-9b7c-e5d5b1a1760e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:42.921784 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acddb4f9-d780-4320-9b7c-e5d5b1a1760e None admin] PATH_INFO: `/v3/projects/155180ba63c446ca889a6eb3f94cc31a/users/1dd72f952df34ad8853fe84673a41f45/roles/5f27c72acb19431fa3223c962bc42938` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:42.933820 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9bc33a-295f-40af-9dcf-bc3cdcc1a42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:42.934472 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9bc33a-295f-40af-9dcf-bc3cdcc1a42c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:42.934754 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9bc33a-295f-40af-9dcf-bc3cdcc1a42c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:42.935027 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9bc33a-295f-40af-9dcf-bc3cdcc1a42c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:42.935855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9bc33a-295f-40af-9dcf-bc3cdcc1a42c None admin] RBAC: Adding query filter params (role.id=5f27c72acb19431fa3223c962bc42938, scope.project.id=155180ba63c446ca889a6eb3f94cc31a, user.id=96db03079816461bb322633f64b9fb9d) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:42.936165 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9bc33a-295f-40af-9dcf-bc3cdcc1a42c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:42.937886 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9bc33a-295f-40af-9dcf-bc3cdcc1a42c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:42.938375 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acddb4f9-d780-4320-9b7c-e5d5b1a1760e None admin] RBAC: Authorizing `identity:create_grant(project_id=155180ba63c446ca889a6eb3f94cc31a, user_id=1dd72f952df34ad8853fe84673a41f45, role_id=5f27c72acb19431fa3223c962bc42938)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:42.939989 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acddb4f9-d780-4320-9b7c-e5d5b1a1760e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:42.952332 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 293/593] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:48:42 2022] GET /identity/v3/role_assignments?user.id=96db03079816461bb322633f64b9fb9d&scope.project.id=155180ba63c446ca889a6eb3f94cc31a&role.id=5f27c72acb19431fa3223c962bc42938 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:42.952332 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9bc33a-295f-40af-9dcf-bc3cdcc1a42c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:43.037315 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 301/594] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jun 23 19:48:42 2022] PUT /identity/v3/projects/155180ba63c446ca889a6eb3f94cc31a/users/1dd72f952df34ad8853fe84673a41f45/roles/5f27c72acb19431fa3223c962bc42938 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:48:43.044671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acddb4f9-d780-4320-9b7c-e5d5b1a1760e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:43.081093 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:43.083936 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:43.105486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9bc33a-295f-40af-9dcf-bc3cdcc1a42c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:43.144516 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90e9ed-0ed5-442b-8226-27008f754f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:43.145326 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90e9ed-0ed5-442b-8226-27008f754f8c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:43.145608 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90e9ed-0ed5-442b-8226-27008f754f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:43.146664 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90e9ed-0ed5-442b-8226-27008f754f8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:43.159373 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90e9ed-0ed5-442b-8226-27008f754f8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:43.160937 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90e9ed-0ed5-442b-8226-27008f754f8c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:43.171694 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acddb4f9-d780-4320-9b7c-e5d5b1a1760e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:43.180548 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-5c90e9ed-0ed5-442b-8226-27008f754f8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:43.180548 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:43.180548 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:43.180548 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:43.180548 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:43.180548 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:43.180548 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:43.180548 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:43.180548 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:43.180548 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:43.180548 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:43.180548 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:43.180548 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:43.180548 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:43.180548 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:43.181536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:43.181536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:43.181536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:43.181536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:43.181536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:43.181536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:43.181536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:43.181536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:43.181536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:43.181536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:43.181536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:43.181536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:43.181536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:43.181536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:43.181536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:43.181536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:43.181536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:43.182765 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:43.182765 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:43.182765 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:43.182765 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:43.182765 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 294/595] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:43.187552 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90e9ed-0ed5-442b-8226-27008f754f8c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:43.218171 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f364f628-efee-4dcb-be42-b48e607e71a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:43.219206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364f628-efee-4dcb-be42-b48e607e71a2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:43.219765 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364f628-efee-4dcb-be42-b48e607e71a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:43.220176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f364f628-efee-4dcb-be42-b48e607e71a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:43.221110 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f364f628-efee-4dcb-be42-b48e607e71a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:43.221567 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f364f628-efee-4dcb-be42-b48e607e71a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:43.223320 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f364f628-efee-4dcb-be42-b48e607e71a2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:43.242150 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 302/596] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:43.245543 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f364f628-efee-4dcb-be42-b48e607e71a2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:43.297267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90e9ed-0ed5-442b-8226-27008f754f8c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:43.334803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f0d7e-513e-49b1-80ba-f95bdf44fc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:43.335570 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f0d7e-513e-49b1-80ba-f95bdf44fc4e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:43.336003 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f0d7e-513e-49b1-80ba-f95bdf44fc4e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:43.336377 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f0d7e-513e-49b1-80ba-f95bdf44fc4e None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:43.337304 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f0d7e-513e-49b1-80ba-f95bdf44fc4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:43.337625 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f0d7e-513e-49b1-80ba-f95bdf44fc4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:43.339108 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f0d7e-513e-49b1-80ba-f95bdf44fc4e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:43.348766 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:43.351916 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:43.353015 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 295/597] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:43.358919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f0d7e-513e-49b1-80ba-f95bdf44fc4e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:43.367028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f364f628-efee-4dcb-be42-b48e607e71a2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:43.404948 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49270d-4b4b-48c9-aba6-48de390d4401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:43.405926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49270d-4b4b-48c9-aba6-48de390d4401 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:43.406411 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49270d-4b4b-48c9-aba6-48de390d4401 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:43.406643 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49270d-4b4b-48c9-aba6-48de390d4401 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:43.414872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49270d-4b4b-48c9-aba6-48de390d4401 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:43.416541 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49270d-4b4b-48c9-aba6-48de390d4401 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:43.431558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-ea49270d-4b4b-48c9-aba6-48de390d4401 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 19:48:43.431558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:43.431558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:43.431558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:43.431558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:43.431558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:43.431558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:43.431558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:43.431558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:43.431558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:43.431558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:43.431558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:43.431558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:48:43.431558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:48:43.431558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:48:43.431558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:48:43.432734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:43.432734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:43.432734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:48:43.432734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:43.432734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:48:43.432734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:43.432734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:43.432734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:43.432734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:43.432734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:43.432734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:43.432734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:43.432734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:43.432734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:43.432734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:43.432734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:43.432734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:43.432734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:43.433672 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:43.433672 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:43.433672 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:48:43.433672 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:48:43.433672 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:48:43.433672 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:48:43.433672 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:48:43.433672 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:48:43.433672 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 19:48:43.433672 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:43.434293 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 303/598] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jun 23 19:48:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:48:43.435546 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d86d6-3f17-4292-a9cf-9ca832fce581 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:43.435988 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d86d6-3f17-4292-a9cf-9ca832fce581 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:43.436380 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d86d6-3f17-4292-a9cf-9ca832fce581 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:43.438258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 304/599] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:43.439266 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d86d6-3f17-4292-a9cf-9ca832fce581 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:43.479109 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f0d7e-513e-49b1-80ba-f95bdf44fc4e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:43.531456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513099a1-3616-4616-8e41-4f84f199baa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:43.533863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513099a1-3616-4616-8e41-4f84f199baa6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:43.534548 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513099a1-3616-4616-8e41-4f84f199baa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:43.535397 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513099a1-3616-4616-8e41-4f84f199baa6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:43.546475 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d86d6-3f17-4292-a9cf-9ca832fce581 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:43.548169 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513099a1-3616-4616-8e41-4f84f199baa6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:43.550700 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513099a1-3616-4616-8e41-4f84f199baa6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:43.573344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-513099a1-3616-4616-8e41-4f84f199baa6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:43.573344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:43.573344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:43.573344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:43.573344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:43.573344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:43.573344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:43.573344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:43.573344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:43.573344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:43.573344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:43.573344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:43.573344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:48:43.573344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:48:43.573344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:48:43.574488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:48:43.574488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:43.574488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:43.574488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:43.574488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:43.574488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:43.574488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:43.574488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:43.574488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:43.574488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:43.574488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:43.574488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:43.574488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:43.574488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:43.574488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:43.574488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:43.574488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:43.575602 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:48:43.575602 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:48:43.575602 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:48:43.575602 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:48:43.575602 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:48:43.575602 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:48:43.575602 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:43.575602 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:43.582654 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 296/600] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:48:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:43.582654 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0153c-92b2-4462-bc4e-3bcdad0e3fd9 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:43.582654 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0153c-92b2-4462-bc4e-3bcdad0e3fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:43.582654 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0153c-92b2-4462-bc4e-3bcdad0e3fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:43.590861 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382f80e-c7a9-4c59-920d-21ee5743aa7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:43.591282 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382f80e-c7a9-4c59-920d-21ee5743aa7c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:43.591659 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382f80e-c7a9-4c59-920d-21ee5743aa7c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:43.591996 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382f80e-c7a9-4c59-920d-21ee5743aa7c None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:43.592872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6382f80e-c7a9-4c59-920d-21ee5743aa7c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:43.593183 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6382f80e-c7a9-4c59-920d-21ee5743aa7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:43.594675 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6382f80e-c7a9-4c59-920d-21ee5743aa7c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:43.605622 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-6ec0153c-92b2-4462-bc4e-3bcdad0e3fd9 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:43.607117 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6ec0153c-92b2-4462-bc4e-3bcdad0e3fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:43.613826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 305/601] 10.222.0.44 () {56 vars in 1139 bytes} [Thu Jun 23 19:48:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:43.615216 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382f80e-c7a9-4c59-920d-21ee5743aa7c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:43.629747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:43.677545 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:43.687991 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 297/602] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:43.689179 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec0153c-92b2-4462-bc4e-3bcdad0e3fd9 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:43.717985 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382f80e-c7a9-4c59-920d-21ee5743aa7c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:43.759952 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a38de-d47a-418d-9832-91f905779bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:43.761038 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a38de-d47a-418d-9832-91f905779bad None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:43.761527 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a38de-d47a-418d-9832-91f905779bad None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:43.762003 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a38de-d47a-418d-9832-91f905779bad None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:43.762986 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a38de-d47a-418d-9832-91f905779bad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:43.763495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a38de-d47a-418d-9832-91f905779bad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:43.765176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a38de-d47a-418d-9832-91f905779bad None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:43.782621 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 306/603] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:48:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:43.783789 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a38de-d47a-418d-9832-91f905779bad None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:43.801718 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec0153c-92b2-4462-bc4e-3bcdad0e3fd9 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:43.846882 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa40c5-3307-4275-84bb-15d8c0eeb1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:43.847584 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa40c5-3307-4275-84bb-15d8c0eeb1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:43.847863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa40c5-3307-4275-84bb-15d8c0eeb1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:43.848134 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa40c5-3307-4275-84bb-15d8c0eeb1c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:43.862727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa40c5-3307-4275-84bb-15d8c0eeb1c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:43.865461 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa40c5-3307-4275-84bb-15d8c0eeb1c6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:43.873910 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:43.891437 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-fbaa40c5-3307-4275-84bb-15d8c0eeb1c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:43.891437 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:43.891437 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:43.891437 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:43.891437 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:43.891437 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:43.891437 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:43.891437 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:43.891437 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:43.891437 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:43.891437 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:43.891437 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:43.891437 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:43.891437 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:43.891437 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:43.892415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:43.892415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:43.892415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:43.892415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:43.892415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:43.892415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:43.892415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:43.892415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:43.892415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:43.892415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:43.892415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:43.892415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:43.892415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:43.892415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:43.892415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:43.892415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:43.892415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:43.893298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:43.893298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:43.893298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:43.893298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:43.896387 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 298/604] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:43.897211 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa40c5-3307-4275-84bb-15d8c0eeb1c6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:43.915985 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a38de-d47a-418d-9832-91f905779bad None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:43.950691 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:43.972711 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea68af1-60f1-4967-aff9-7694bf728f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:43.973561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea68af1-60f1-4967-aff9-7694bf728f36 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:43.973803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea68af1-60f1-4967-aff9-7694bf728f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:43.975625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea68af1-60f1-4967-aff9-7694bf728f36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:43.994071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea68af1-60f1-4967-aff9-7694bf728f36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:44.005323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-dea68af1-60f1-4967-aff9-7694bf728f36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:44.005323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:44.005323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:44.005323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:44.005323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:44.005323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:44.005323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:44.005323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:44.005323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:44.005323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:44.005323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:44.005323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:44.005323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:44.005323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:44.005323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:44.005323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:44.006260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:44.006260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:44.006260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:44.006260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:44.006260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:44.006260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:44.006260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:44.006260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:44.006260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:44.006260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:44.006260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:44.006260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:44.006260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:44.006260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:44.006260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:44.006260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:44.007072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:44.007072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:44.007072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:44.007072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:44.007072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:44.007072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:44.007072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:44.007072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:44.007072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:44.007072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:44.007072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:44.007072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:44.009966 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 307/605] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jun 23 19:48:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:44.010911 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea68af1-60f1-4967-aff9-7694bf728f36 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:44.031444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa40c5-3307-4275-84bb-15d8c0eeb1c6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:44.079513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770c60b-6e3c-4d1f-8805-09bfbdaf3df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:44.080915 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770c60b-6e3c-4d1f-8805-09bfbdaf3df8 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:44.081360 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770c60b-6e3c-4d1f-8805-09bfbdaf3df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:44.081844 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d770c60b-6e3c-4d1f-8805-09bfbdaf3df8 None admin] PATH_INFO: `/v3/projects/155180ba63c446ca889a6eb3f94cc31a/users/21ae3b8971b44d928e9f6c0539858636/roles/5f27c72acb19431fa3223c962bc42938` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:44.102064 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d770c60b-6e3c-4d1f-8805-09bfbdaf3df8 None admin] RBAC: Authorizing `identity:create_grant(project_id=155180ba63c446ca889a6eb3f94cc31a, user_id=21ae3b8971b44d928e9f6c0539858636, role_id=5f27c72acb19431fa3223c962bc42938)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:44.104583 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d770c60b-6e3c-4d1f-8805-09bfbdaf3df8 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:44.119947 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea68af1-60f1-4967-aff9-7694bf728f36 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:44.149698 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:44.155513 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e884f-0251-4338-9cf7-720098aef84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:44.156244 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e884f-0251-4338-9cf7-720098aef84f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:44.156562 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e884f-0251-4338-9cf7-720098aef84f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:44.156831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e884f-0251-4338-9cf7-720098aef84f None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:44.157544 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e884f-0251-4338-9cf7-720098aef84f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:44.157852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e884f-0251-4338-9cf7-720098aef84f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:44.159324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e884f-0251-4338-9cf7-720098aef84f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:44.172386 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 308/606] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:44.173559 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e884f-0251-4338-9cf7-720098aef84f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:44.232331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 299/607] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jun 23 19:48:43 2022] PUT /identity/v3/projects/155180ba63c446ca889a6eb3f94cc31a/users/21ae3b8971b44d928e9f6c0539858636/roles/5f27c72acb19431fa3223c962bc42938 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:48:44.232331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770c60b-6e3c-4d1f-8805-09bfbdaf3df8 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:44.349428 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e884f-0251-4338-9cf7-720098aef84f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:44.391338 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a606ae-9dce-46b4-a140-e7970a05b194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:44.391969 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a606ae-9dce-46b4-a140-e7970a05b194 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:44.392259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a606ae-9dce-46b4-a140-e7970a05b194 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:44.392628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a606ae-9dce-46b4-a140-e7970a05b194 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:44.394413 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a606ae-9dce-46b4-a140-e7970a05b194 None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:44.394413 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a606ae-9dce-46b4-a140-e7970a05b194 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:44.395572 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a606ae-9dce-46b4-a140-e7970a05b194 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:44.407161 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 309/608] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jun 23 19:48:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:44.414570 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a606ae-9dce-46b4-a140-e7970a05b194 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:44.426970 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:44.451219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d770c60b-6e3c-4d1f-8805-09bfbdaf3df8 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:44.478910 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:44.505303 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da1471-cff3-453a-aff2-36e7620956ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:44.506462 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da1471-cff3-453a-aff2-36e7620956ab None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:44.506956 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da1471-cff3-453a-aff2-36e7620956ab None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:44.507450 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da1471-cff3-453a-aff2-36e7620956ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:44.515103 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da1471-cff3-453a-aff2-36e7620956ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:44.516985 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da1471-cff3-453a-aff2-36e7620956ab None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:44.531106 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-f8da1471-cff3-453a-aff2-36e7620956ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:44.531106 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:44.531106 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:44.531106 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:44.531106 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:44.531106 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:44.531106 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:44.531106 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:44.531106 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:44.531106 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:44.531106 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:44.531106 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:44.531106 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:48:44.531106 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:48:44.531106 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:48:44.532023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:48:44.532023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:44.532023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:44.532023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:44.532023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:44.532023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:44.532023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:44.532023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:44.532023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:44.532023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:44.532023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:44.532023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:44.532023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:44.532023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:44.532023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:44.532023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:44.532023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:44.533776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:48:44.533776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:48:44.533776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:48:44.533776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:48:44.533776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:48:44.533776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:48:44.533776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:44.533776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:44.534204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a606ae-9dce-46b4-a140-e7970a05b194 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:44.536083 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 300/609] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:48:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:44.542476 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da1471-cff3-453a-aff2-36e7620956ab None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:44.577060 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5de61-ec60-4dd9-80dd-e1a18fbc0d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:44.578164 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5de61-ec60-4dd9-80dd-e1a18fbc0d84 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:44.578449 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5de61-ec60-4dd9-80dd-e1a18fbc0d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:44.578735 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5de61-ec60-4dd9-80dd-e1a18fbc0d84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:44.592842 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e5de61-ec60-4dd9-80dd-e1a18fbc0d84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:44.594317 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e5de61-ec60-4dd9-80dd-e1a18fbc0d84 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:44.616027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-50e5de61-ec60-4dd9-80dd-e1a18fbc0d84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:44.616027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:44.616027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:44.616027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:44.616027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:44.616027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:44.616027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:44.616027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:44.616027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:44.616027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:44.616027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:44.616027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:44.616027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:44.616027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:44.616027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:44.617034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:44.617034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:44.617034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:44.617034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:44.617034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:44.617034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:44.617034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:44.617034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:44.617034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:44.617034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:44.617034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:44.617034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:44.617034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:44.617034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:44.617034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:44.617034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:44.617034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:44.618573 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:44.618573 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:44.618573 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:44.618573 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:44.618573 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 310/610] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:48:44.622786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5de61-ec60-4dd9-80dd-e1a18fbc0d84 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:44.666142 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da1471-cff3-453a-aff2-36e7620956ab None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:44.719473 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a8f38d-6525-4008-ab69-6789e5943faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:44.721285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8f38d-6525-4008-ab69-6789e5943faa None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:44.721576 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8f38d-6525-4008-ab69-6789e5943faa None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:44.721853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8f38d-6525-4008-ab69-6789e5943faa None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:44.722643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a8f38d-6525-4008-ab69-6789e5943faa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:44.722952 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a8f38d-6525-4008-ab69-6789e5943faa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:44.724490 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a8f38d-6525-4008-ab69-6789e5943faa None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:44.746999 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 301/611] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:48:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:44.752643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a8f38d-6525-4008-ab69-6789e5943faa None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:44.781796 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:44.789800 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5de61-ec60-4dd9-80dd-e1a18fbc0d84 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:44.795015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:44.856862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2002e-0322-42b6-822b-eb8b13375515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:44.857911 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2002e-0322-42b6-822b-eb8b13375515 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:44.858400 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2002e-0322-42b6-822b-eb8b13375515 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:44.858895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2002e-0322-42b6-822b-eb8b13375515 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:44.859831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2002e-0322-42b6-822b-eb8b13375515 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:44.860399 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2002e-0322-42b6-822b-eb8b13375515 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:44.861978 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2002e-0322-42b6-822b-eb8b13375515 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:44.894657 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 311/612] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:44.909912 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2002e-0322-42b6-822b-eb8b13375515 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:44.933868 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a8f38d-6525-4008-ab69-6789e5943faa None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:44.975895 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c491f3-9a7a-4c7d-9cc4-7b14aad4517e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:44.977685 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c491f3-9a7a-4c7d-9cc4-7b14aad4517e None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:44.977969 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c491f3-9a7a-4c7d-9cc4-7b14aad4517e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:44.978242 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c491f3-9a7a-4c7d-9cc4-7b14aad4517e None admin] PATH_INFO: `/v3/projects/155180ba63c446ca889a6eb3f94cc31a/users/332fbbefba114da4a01fdd496fff1e31/roles/5f27c72acb19431fa3223c962bc42938` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:45.011771 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c491f3-9a7a-4c7d-9cc4-7b14aad4517e None admin] RBAC: Authorizing `identity:create_grant(project_id=155180ba63c446ca889a6eb3f94cc31a, user_id=332fbbefba114da4a01fdd496fff1e31, role_id=5f27c72acb19431fa3223c962bc42938)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:45.013283 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c491f3-9a7a-4c7d-9cc4-7b14aad4517e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:45.076915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2002e-0322-42b6-822b-eb8b13375515 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:45.120934 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 302/613] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jun 23 19:48:44 2022] PUT /identity/v3/projects/155180ba63c446ca889a6eb3f94cc31a/users/332fbbefba114da4a01fdd496fff1e31/roles/5f27c72acb19431fa3223c962bc42938 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:48:45.126491 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384f223-edc9-469f-9f83-3b9536421efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:45.127354 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384f223-edc9-469f-9f83-3b9536421efd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:45.127642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384f223-edc9-469f-9f83-3b9536421efd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:45.127984 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384f223-edc9-469f-9f83-3b9536421efd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:45.137045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a384f223-edc9-469f-9f83-3b9536421efd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:45.138550 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a384f223-edc9-469f-9f83-3b9536421efd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:45.160324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-a384f223-edc9-469f-9f83-3b9536421efd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 19:48:45.160324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:45.160324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:45.160324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:45.160324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:45.160324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:45.160324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:45.160324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:45.160324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:45.160324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:45.160324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:45.160324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:45.160324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:48:45.160324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:48:45.160324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:48:45.160324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:48:45.161882 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:45.161882 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:45.161882 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:48:45.161882 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:45.161882 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:48:45.161882 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:45.161882 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:45.161882 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:45.161882 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:45.161882 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:45.161882 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:45.161882 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:45.161882 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:45.161882 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:45.161882 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:45.161882 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:45.161882 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:45.161882 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:45.162836 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:45.162836 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:45.162836 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:48:45.162836 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:48:45.162836 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:48:45.162836 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:48:45.162836 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:48:45.162836 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:48:45.162836 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 19:48:45.162836 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:45.170231 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 312/614] 10.222.0.44 () {56 vars in 1093 bytes} [Thu Jun 23 19:48:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:45.176122 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c491f3-9a7a-4c7d-9cc4-7b14aad4517e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:45.205063 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00dac9c-a587-4f7b-bc50-694c35e7d756 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:45.205381 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00dac9c-a587-4f7b-bc50-694c35e7d756 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:45.205539 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00dac9c-a587-4f7b-bc50-694c35e7d756 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:45.207157 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 313/615] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:45.212689 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f5549-c17d-4cfa-9c32-035459c08d11 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:45.212963 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f5549-c17d-4cfa-9c32-035459c08d11 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:45.213233 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f5549-c17d-4cfa-9c32-035459c08d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:45.234991 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-fe0f5549-c17d-4cfa-9c32-035459c08d11 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:45.236213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fe0f5549-c17d-4cfa-9c32-035459c08d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:45.263892 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:45.318504 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:45.346964 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 314/616] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:45.368950 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c491f3-9a7a-4c7d-9cc4-7b14aad4517e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:45.374996 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f5549-c17d-4cfa-9c32-035459c08d11 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:45.409714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04d608-634b-431a-9bc0-ab9676a4bcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:45.410440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04d608-634b-431a-9bc0-ab9676a4bcc4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:45.410728 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04d608-634b-431a-9bc0-ab9676a4bcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:45.411007 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04d608-634b-431a-9bc0-ab9676a4bcc4 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:45.412428 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d04d608-634b-431a-9bc0-ab9676a4bcc4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:45.412744 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d04d608-634b-431a-9bc0-ab9676a4bcc4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:45.414310 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d04d608-634b-431a-9bc0-ab9676a4bcc4 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:45.433332 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 303/617] 10.222.0.44 () {56 vars in 1133 bytes} [Thu Jun 23 19:48:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:45.439173 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04d608-634b-431a-9bc0-ab9676a4bcc4 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:45.501546 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f5549-c17d-4cfa-9c32-035459c08d11 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:45.537927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25debd8-f782-493e-9a50-0f2aff39b09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:45.538733 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25debd8-f782-493e-9a50-0f2aff39b09f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:45.539017 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25debd8-f782-493e-9a50-0f2aff39b09f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:45.539287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25debd8-f782-493e-9a50-0f2aff39b09f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:45.544914 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04d608-634b-431a-9bc0-ab9676a4bcc4 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:45.549726 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25debd8-f782-493e-9a50-0f2aff39b09f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:45.557415 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-f25debd8-f782-493e-9a50-0f2aff39b09f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:45.557415 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:45.557415 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:45.557415 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:45.557415 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:45.557415 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:45.557415 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:45.557415 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:45.557415 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:45.557415 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:45.557415 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:45.557415 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:45.557415 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:45.557415 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:45.557415 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:45.557415 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:45.558322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:45.558322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:45.558322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:45.558322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:45.558322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:45.558322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:45.558322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:45.558322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:45.558322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:45.558322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:45.558322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:45.558322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:45.558322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:45.558322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:45.558322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:45.558322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:45.559301 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:45.559301 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:45.559301 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:45.559301 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:45.559301 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:45.559301 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:45.559301 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:45.559301 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:45.559301 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:45.559301 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:45.559301 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:45.559301 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:45.559301 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 315/618] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jun 23 19:48:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:48:45.560551 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc172b1c-c986-435e-9b53-cb0180a95a76 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:45.560551 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc172b1c-c986-435e-9b53-cb0180a95a76 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:45.560788 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc172b1c-c986-435e-9b53-cb0180a95a76 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:45.562321 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 316/619] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:45.567577 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc172b1c-c986-435e-9b53-cb0180a95a76 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:45.568245 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:45.587007 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8facbd8d-896f-44b7-b829-e580351cab82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:45.587654 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facbd8d-896f-44b7-b829-e580351cab82 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:45.587940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facbd8d-896f-44b7-b829-e580351cab82 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:45.588204 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facbd8d-896f-44b7-b829-e580351cab82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:45.590235 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:45.670611 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8facbd8d-896f-44b7-b829-e580351cab82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:45.672668 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8facbd8d-896f-44b7-b829-e580351cab82 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:45.691371 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc172b1c-c986-435e-9b53-cb0180a95a76 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:45.726821 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17079d31-7f1c-47ab-900b-aa485965d7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:45.727517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17079d31-7f1c-47ab-900b-aa485965d7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:45.727797 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17079d31-7f1c-47ab-900b-aa485965d7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:45.728067 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17079d31-7f1c-47ab-900b-aa485965d7c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:45.728948 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17079d31-7f1c-47ab-900b-aa485965d7c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:45.729256 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17079d31-7f1c-47ab-900b-aa485965d7c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:45.730743 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17079d31-7f1c-47ab-900b-aa485965d7c6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:45.739873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 317/620] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jun 23 19:48:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:45.741242 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb1a81-0d0d-437b-a17f-d2ca6c1ff86d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:45.741609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb1a81-0d0d-437b-a17f-d2ca6c1ff86d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:45.741800 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb1a81-0d0d-437b-a17f-d2ca6c1ff86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:45.758782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-47cb1a81-0d0d-437b-a17f-d2ca6c1ff86d None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:45.759901 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-47cb1a81-0d0d-437b-a17f-d2ca6c1ff86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:45.795823 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 304/621] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:48:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 19:48:45.797221 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8facbd8d-896f-44b7-b829-e580351cab82 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:45.827889 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:45.840574 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 318/622] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:45.841543 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb1a81-0d0d-437b-a17f-d2ca6c1ff86d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:45.854855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:45.929180 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8facbd8d-896f-44b7-b829-e580351cab82 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:45.962005 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb1a81-0d0d-437b-a17f-d2ca6c1ff86d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:45.969825 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9e260-a7c3-459c-bb74-f193479f93cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:45.970623 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9e260-a7c3-459c-bb74-f193479f93cd None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:45.970904 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9e260-a7c3-459c-bb74-f193479f93cd None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:45.971177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9e260-a7c3-459c-bb74-f193479f93cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:45.981734 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f9e260-a7c3-459c-bb74-f193479f93cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:45.983225 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f9e260-a7c3-459c-bb74-f193479f93cd None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:46.003644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-f2f9e260-a7c3-459c-bb74-f193479f93cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:46.003644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:46.003644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:46.003644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:46.003644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:46.003644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:46.003644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:46.003644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:46.003644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:46.003644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:46.003644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:46.003644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:46.003644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:46.003644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:46.003644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:46.004644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:46.004644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:46.004644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:46.004644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:46.004644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:46.004644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:46.004644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:46.004644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:46.004644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:46.004644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:46.004644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:46.004644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:46.004644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:46.004644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:46.004644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:46.004644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:46.004644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:46.006080 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:46.006080 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:46.006080 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:46.006080 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:46.010302 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 305/623] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:46.011176 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9e260-a7c3-459c-bb74-f193479f93cd None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:46.013874 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1314fd-b0ca-4d11-8f24-62c2d152102b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:46.014874 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1314fd-b0ca-4d11-8f24-62c2d152102b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:46.016043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1314fd-b0ca-4d11-8f24-62c2d152102b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:46.016347 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1314fd-b0ca-4d11-8f24-62c2d152102b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:46.030169 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1314fd-b0ca-4d11-8f24-62c2d152102b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:46.031902 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1314fd-b0ca-4d11-8f24-62c2d152102b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:46.052679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-1d1314fd-b0ca-4d11-8f24-62c2d152102b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:46.052679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:46.052679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:46.052679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:46.052679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:46.052679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:46.052679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:46.052679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:46.052679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:46.052679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:46.052679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:46.052679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:46.052679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:46.052679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:46.052679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:46.053781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:46.053781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:46.053781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:46.053781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:46.053781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:46.053781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:46.053781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:46.053781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:46.053781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:46.053781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:46.053781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:46.053781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:46.053781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:46.053781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:46.053781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:46.053781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:46.053781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:46.054913 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:46.054913 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:46.054913 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:46.054913 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:46.056391 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 319/624] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:46.057245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1314fd-b0ca-4d11-8f24-62c2d152102b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:46.099570 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:46.124672 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:46.140741 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9e260-a7c3-459c-bb74-f193479f93cd None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:46.158906 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1314fd-b0ca-4d11-8f24-62c2d152102b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:46.185671 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd9c6b-b75a-4fac-9b12-c16c7926d36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:46.186875 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd9c6b-b75a-4fac-9b12-c16c7926d36a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:46.187172 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd9c6b-b75a-4fac-9b12-c16c7926d36a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:46.187447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd9c6b-b75a-4fac-9b12-c16c7926d36a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:46.198877 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c4e5c-1416-48fe-9aac-5a1ccdf78ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:46.199845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c4e5c-1416-48fe-9aac-5a1ccdf78ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:46.200367 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c4e5c-1416-48fe-9aac-5a1ccdf78ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:46.200849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd9c6b-b75a-4fac-9b12-c16c7926d36a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:46.201038 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c4e5c-1416-48fe-9aac-5a1ccdf78ae8 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:46.201948 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c4e5c-1416-48fe-9aac-5a1ccdf78ae8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:46.202444 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c4e5c-1416-48fe-9aac-5a1ccdf78ae8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:46.203992 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c4e5c-1416-48fe-9aac-5a1ccdf78ae8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:46.209471 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-73cd9c6b-b75a-4fac-9b12-c16c7926d36a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:46.209471 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:46.209471 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:46.209471 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:46.209471 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:46.209471 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:46.209471 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:46.209471 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:46.209471 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:46.209471 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:46.209471 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:46.209471 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:46.209471 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:46.209471 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:46.209471 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:46.209471 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:46.210386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:46.210386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:46.210386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:46.210386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:46.210386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:46.210386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:46.210386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:46.210386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:46.210386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:46.210386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:46.210386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:46.210386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:46.210386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:46.210386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:46.210386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:46.210386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:46.211194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:46.211194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:46.211194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:46.211194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:46.211194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:46.211194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:46.211194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:46.211194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:46.211194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:46.211194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:46.211194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:46.211194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:46.215872 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 306/625] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jun 23 19:48:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:46.216840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd9c6b-b75a-4fac-9b12-c16c7926d36a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:46.226256 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 320/626] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:46.227405 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c4e5c-1416-48fe-9aac-5a1ccdf78ae8 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:46.332347 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c4e5c-1416-48fe-9aac-5a1ccdf78ae8 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:46.332575 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd9c6b-b75a-4fac-9b12-c16c7926d36a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:46.350745 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:46.363008 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:46.369076 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb758e93-fa28-40a5-81c0-c160386894c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:46.369742 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c4f6d8-eb04-4813-970a-938e612623bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:46.370159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb758e93-fa28-40a5-81c0-c160386894c7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:46.370583 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb758e93-fa28-40a5-81c0-c160386894c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:46.370696 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4f6d8-eb04-4813-970a-938e612623bb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:46.371043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4f6d8-eb04-4813-970a-938e612623bb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:46.371258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb758e93-fa28-40a5-81c0-c160386894c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:46.371498 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4f6d8-eb04-4813-970a-938e612623bb None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:46.372290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb758e93-fa28-40a5-81c0-c160386894c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:46.372424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c4f6d8-eb04-4813-970a-938e612623bb None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:46.372825 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c4f6d8-eb04-4813-970a-938e612623bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:46.373053 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb758e93-fa28-40a5-81c0-c160386894c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:46.374365 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c4f6d8-eb04-4813-970a-938e612623bb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:46.374728 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb758e93-fa28-40a5-81c0-c160386894c7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:46.388352 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 321/627] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jun 23 19:48:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:46.388352 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c4f6d8-eb04-4813-970a-938e612623bb None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:46.393213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 307/628] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:46.394190 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb758e93-fa28-40a5-81c0-c160386894c7 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:46.496835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c4f6d8-eb04-4813-970a-938e612623bb None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:46.504459 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb758e93-fa28-40a5-81c0-c160386894c7 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:46.531774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cdd147-1b8d-4a03-b2ea-95a7b571ee92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:46.532685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdd147-1b8d-4a03-b2ea-95a7b571ee92 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:46.533037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdd147-1b8d-4a03-b2ea-95a7b571ee92 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:46.533395 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cdd147-1b8d-4a03-b2ea-95a7b571ee92 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:46.539492 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3190f62-9410-453a-b6fd-28ab3d3e56e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:46.540271 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3190f62-9410-453a-b6fd-28ab3d3e56e7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:46.540599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3190f62-9410-453a-b6fd-28ab3d3e56e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:46.540845 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3190f62-9410-453a-b6fd-28ab3d3e56e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:46.542108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cdd147-1b8d-4a03-b2ea-95a7b571ee92 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:46.543621 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cdd147-1b8d-4a03-b2ea-95a7b571ee92 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:46.553237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3190f62-9410-453a-b6fd-28ab3d3e56e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:46.554701 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3190f62-9410-453a-b6fd-28ab3d3e56e7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:46.558417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-c3cdd147-1b8d-4a03-b2ea-95a7b571ee92 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 19:48:46.558417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:46.558417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:46.558417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:46.558417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:46.558417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:46.558417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:46.558417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:46.558417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:46.558417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:46.558417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:46.558417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:46.558417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:48:46.558417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:48:46.558417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:48:46.558417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:48:46.559331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:46.559331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:46.559331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:48:46.559331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:46.559331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:48:46.559331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:46.559331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:46.559331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:46.559331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:46.559331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:46.559331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:46.559331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:46.559331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:46.559331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:46.559331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:46.559331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:46.559331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:46.559331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:46.560270 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:46.560270 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:46.560270 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:48:46.560270 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:48:46.560270 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:48:46.560270 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:48:46.560270 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:48:46.560270 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:48:46.560270 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 19:48:46.560270 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:46.564528 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 322/629] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:48:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:46.565400 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cdd147-1b8d-4a03-b2ea-95a7b571ee92 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:46.575037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-d3190f62-9410-453a-b6fd-28ab3d3e56e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:46.575037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:46.575037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:46.575037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:46.575037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:46.575037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:46.575037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:46.575037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:46.575037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:46.575037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:46.575037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:46.575037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:46.575037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:46.575037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:46.575037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:46.576767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:46.576767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:46.576767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:46.576767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:46.576767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:46.576767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:46.576767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:46.576767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:46.576767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:46.576767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:46.576767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:46.576767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:46.576767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:46.576767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:46.576767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:46.576767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:46.576767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:46.577931 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:46.577931 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:46.577931 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:46.577931 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:46.580836 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 308/630] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:46.581261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3190f62-9410-453a-b6fd-28ab3d3e56e7 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:46.614219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:46.644372 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:46.675372 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cdd147-1b8d-4a03-b2ea-95a7b571ee92 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:46.712507 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3190f62-9410-453a-b6fd-28ab3d3e56e7 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:46.713058 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2993eae-ef78-4fff-8559-fbbeed8a63db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:46.713946 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2993eae-ef78-4fff-8559-fbbeed8a63db None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:46.714324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2993eae-ef78-4fff-8559-fbbeed8a63db None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:46.714787 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2993eae-ef78-4fff-8559-fbbeed8a63db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:46.722036 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2993eae-ef78-4fff-8559-fbbeed8a63db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:46.723600 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2993eae-ef78-4fff-8559-fbbeed8a63db None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:46.740287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-c2993eae-ef78-4fff-8559-fbbeed8a63db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:46.740287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:46.740287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:46.740287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:46.740287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:46.740287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:46.740287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:46.740287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:46.740287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:46.740287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:46.740287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:46.740287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:46.740287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:48:46.740287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:48:46.740287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:48:46.741398 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:48:46.741398 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:46.741398 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:46.741398 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:46.741398 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:46.741398 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:46.741398 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:46.741398 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:46.741398 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:46.741398 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:46.741398 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:46.741398 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:46.741398 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:46.741398 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:46.741398 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:46.741398 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:46.741398 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:46.742487 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:48:46.742487 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:48:46.742487 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:48:46.742487 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:48:46.742487 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:48:46.742487 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:48:46.742487 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:46.742487 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:46.744223 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 323/631] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:48:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:46.745275 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2993eae-ef78-4fff-8559-fbbeed8a63db None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:46.755556 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da039e-a3b1-4aec-abe3-8bb50443acfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:46.756821 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da039e-a3b1-4aec-abe3-8bb50443acfc None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:46.757100 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da039e-a3b1-4aec-abe3-8bb50443acfc None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:46.757370 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da039e-a3b1-4aec-abe3-8bb50443acfc None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:46.758208 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da039e-a3b1-4aec-abe3-8bb50443acfc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:46.758517 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da039e-a3b1-4aec-abe3-8bb50443acfc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:46.759958 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da039e-a3b1-4aec-abe3-8bb50443acfc None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:46.783146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 309/632] 10.222.0.44 () {56 vars in 1135 bytes} [Thu Jun 23 19:48:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:46.784132 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da039e-a3b1-4aec-abe3-8bb50443acfc None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:46.859910 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:46.863066 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2993eae-ef78-4fff-8559-fbbeed8a63db None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:46.889581 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da039e-a3b1-4aec-abe3-8bb50443acfc None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:46.891145 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:46.897542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5e054-0d13-4faa-87ad-2ce477ed0e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:46.898214 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5e054-0d13-4faa-87ad-2ce477ed0e5e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:46.898914 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5e054-0d13-4faa-87ad-2ce477ed0e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:46.899188 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5e054-0d13-4faa-87ad-2ce477ed0e5e None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:46.899918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5e054-0d13-4faa-87ad-2ce477ed0e5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:46.900414 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5e054-0d13-4faa-87ad-2ce477ed0e5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:46.902362 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5e054-0d13-4faa-87ad-2ce477ed0e5e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:46.918490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 324/633] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:46.918490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5e054-0d13-4faa-87ad-2ce477ed0e5e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:46.929368 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c2f3e-cbf7-4b0c-bc4c-88a099042460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:46.930028 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c2f3e-cbf7-4b0c-bc4c-88a099042460 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:46.930306 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c2f3e-cbf7-4b0c-bc4c-88a099042460 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:46.930579 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c2f3e-cbf7-4b0c-bc4c-88a099042460 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:46.931356 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c2f3e-cbf7-4b0c-bc4c-88a099042460 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:46.931660 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c2f3e-cbf7-4b0c-bc4c-88a099042460 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:46.933162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c2f3e-cbf7-4b0c-bc4c-88a099042460 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:46.950488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 310/634] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:48:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:46.950488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cb67c-d127-4294-912b-1d7dba89493a None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:46.950488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cb67c-d127-4294-912b-1d7dba89493a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:46.950488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cb67c-d127-4294-912b-1d7dba89493a None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:46.950488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 311/635] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:46.950488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140cb67c-d127-4294-912b-1d7dba89493a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:47.024154 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5e054-0d13-4faa-87ad-2ce477ed0e5e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:47.055657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140cb67c-d127-4294-912b-1d7dba89493a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:47.059208 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682925ad-9fd5-45e0-bb95-8c003645eb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:47.059986 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682925ad-9fd5-45e0-bb95-8c003645eb54 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:47.060268 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682925ad-9fd5-45e0-bb95-8c003645eb54 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:47.060578 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682925ad-9fd5-45e0-bb95-8c003645eb54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:47.071813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682925ad-9fd5-45e0-bb95-8c003645eb54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:47.073395 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682925ad-9fd5-45e0-bb95-8c003645eb54 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:47.090328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099135a-bd73-4551-acfa-17de642e09de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:47.090804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-682925ad-9fd5-45e0-bb95-8c003645eb54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:47.090804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:47.090804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:47.090804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:47.090804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:47.090804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:47.090804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:47.090804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:47.090804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:47.090804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:47.090804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:47.090804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:47.090804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:47.090804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:47.090804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:47.091663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:47.091663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:47.091663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:47.091663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:47.091663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:47.091663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:47.091663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:47.091663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:47.091663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:47.091663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:47.091663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:47.091663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:47.091663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:47.091663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:47.091663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:47.091663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:47.091663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:47.097293 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:47.097293 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:47.097293 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:47.097293 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:47.097511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099135a-bd73-4551-acfa-17de642e09de None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:47.097511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099135a-bd73-4551-acfa-17de642e09de None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:47.097511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099135a-bd73-4551-acfa-17de642e09de None admin] PATH_INFO: `/v3/users/glance` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:47.097703 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 325/636] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:47.097703 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682925ad-9fd5-45e0-bb95-8c003645eb54 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:47.106160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099135a-bd73-4551-acfa-17de642e09de None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:47.107633 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099135a-bd73-4551-acfa-17de642e09de None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:47.112126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:47.122776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-2099135a-bd73-4551-acfa-17de642e09de None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 19:48:47.122776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:47.122776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:47.122776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:47.122776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:47.122776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:47.122776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:47.122776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:47.122776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:47.122776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:47.122776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:47.122776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:47.122776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:48:47.122776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:48:47.122776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:48:47.122776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:48:47.123855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:47.123855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:47.123855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:48:47.123855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:47.123855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:48:47.123855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:47.123855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:47.123855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:47.123855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:47.123855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:47.123855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:47.123855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:47.123855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:47.123855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:47.123855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:47.123855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:47.123855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:47.123855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:47.129596 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:47.129596 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:47.129596 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:48:47.129596 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:48:47.129596 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:48:47.129596 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:48:47.129596 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:48:47.129596 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:48:47.129596 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 19:48:47.129596 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:47.129596 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 312/637] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:47.129596 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726352f-2b6e-46c8-9729-1ab44ccb5de8 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:47.129596 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726352f-2b6e-46c8-9729-1ab44ccb5de8 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:47.130890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726352f-2b6e-46c8-9729-1ab44ccb5de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:47.142969 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f726352f-2b6e-46c8-9729-1ab44ccb5de8 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:47.144092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f726352f-2b6e-46c8-9729-1ab44ccb5de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:47.160799 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:47.205450 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682925ad-9fd5-45e0-bb95-8c003645eb54 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:47.214452 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 313/638] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:47.215443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726352f-2b6e-46c8-9729-1ab44ccb5de8 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:47.263628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f230109-7c26-44e1-80f4-194cdea7bb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:47.264827 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f230109-7c26-44e1-80f4-194cdea7bb1c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:47.265114 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f230109-7c26-44e1-80f4-194cdea7bb1c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:47.265395 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f230109-7c26-44e1-80f4-194cdea7bb1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:47.266675 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f230109-7c26-44e1-80f4-194cdea7bb1c None admin] RBAC: Adding query filter params (role.id=5f27c72acb19431fa3223c962bc42938, scope.project.id=155180ba63c446ca889a6eb3f94cc31a, user.id=a63d1d902c024c97a38f7d92f73814d3) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:47.266996 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f230109-7c26-44e1-80f4-194cdea7bb1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:47.269173 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f230109-7c26-44e1-80f4-194cdea7bb1c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:47.284720 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 326/639] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:48:47 2022] GET /identity/v3/role_assignments?user.id=a63d1d902c024c97a38f7d92f73814d3&scope.project.id=155180ba63c446ca889a6eb3f94cc31a&role.id=5f27c72acb19431fa3223c962bc42938 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:47.285536 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f230109-7c26-44e1-80f4-194cdea7bb1c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:47.340918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726352f-2b6e-46c8-9729-1ab44ccb5de8 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:47.386710 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9bf6a-7af6-4644-bac4-f61db19d36ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:47.387885 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9bf6a-7af6-4644-bac4-f61db19d36ec None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:47.388237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9bf6a-7af6-4644-bac4-f61db19d36ec None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:47.388364 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:47.388630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9bf6a-7af6-4644-bac4-f61db19d36ec None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:47.389452 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9bf6a-7af6-4644-bac4-f61db19d36ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:47.389833 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9bf6a-7af6-4644-bac4-f61db19d36ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:47.391339 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9bf6a-7af6-4644-bac4-f61db19d36ec None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:47.408635 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 314/640] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:47.408635 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9bf6a-7af6-4644-bac4-f61db19d36ec None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:47.418638 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f230109-7c26-44e1-80f4-194cdea7bb1c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:47.426274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:47.460405 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb71e4-f6e8-4639-8941-1717adbd4f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:47.460909 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb71e4-f6e8-4639-8941-1717adbd4f3f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:47.461191 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb71e4-f6e8-4639-8941-1717adbd4f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:47.461477 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb71e4-f6e8-4639-8941-1717adbd4f3f None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:47.462313 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb71e4-f6e8-4639-8941-1717adbd4f3f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:47.462627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb71e4-f6e8-4639-8941-1717adbd4f3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:47.464060 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb71e4-f6e8-4639-8941-1717adbd4f3f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:47.485429 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 327/641] 10.222.0.44 () {56 vars in 1137 bytes} [Thu Jun 23 19:48:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:47.486724 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff45a9c-9987-40a7-83c9-6fe6e88cf425 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:47.487012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff45a9c-9987-40a7-83c9-6fe6e88cf425 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:47.487291 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff45a9c-9987-40a7-83c9-6fe6e88cf425 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:47.489811 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 328/642] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:47.490321 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff45a9c-9987-40a7-83c9-6fe6e88cf425 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:47.527359 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9bf6a-7af6-4644-bac4-f61db19d36ec None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:47.562893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141babea-865b-454a-ba34-b9c8d3eba666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:47.563686 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141babea-865b-454a-ba34-b9c8d3eba666 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:47.563962 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141babea-865b-454a-ba34-b9c8d3eba666 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:47.564239 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141babea-865b-454a-ba34-b9c8d3eba666 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:47.574681 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141babea-865b-454a-ba34-b9c8d3eba666 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:47.582284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-141babea-865b-454a-ba34-b9c8d3eba666 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:47.582284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:47.582284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:47.582284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:47.582284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:47.582284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:47.582284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:47.582284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:47.582284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:47.582284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:47.582284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:47.582284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:47.582284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:47.582284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:47.582284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:47.582284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:47.584058 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:47.584058 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:47.584058 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:47.584058 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:47.584058 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:47.584058 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:47.584058 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:47.584058 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:47.584058 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:47.584058 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:47.584058 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:47.584058 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:47.584058 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:47.584058 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:47.584058 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:47.584058 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:47.584987 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:47.584987 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:47.584987 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:47.584987 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:47.584987 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:47.584987 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:47.584987 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:47.584987 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:47.584987 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:47.584987 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:47.584987 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:47.584987 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:47.584987 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 315/643] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jun 23 19:48:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:47.585697 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141babea-865b-454a-ba34-b9c8d3eba666 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:47.606987 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff45a9c-9987-40a7-83c9-6fe6e88cf425 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:47.643376 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf5ba4-36b0-45f5-bb17-ece1135a17b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:47.644564 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf5ba4-36b0-45f5-bb17-ece1135a17b8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:47.645137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf5ba4-36b0-45f5-bb17-ece1135a17b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:47.645418 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf5ba4-36b0-45f5-bb17-ece1135a17b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:47.648099 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:47.653606 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf5ba4-36b0-45f5-bb17-ece1135a17b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:47.655253 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf5ba4-36b0-45f5-bb17-ece1135a17b8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:47.661286 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:47.669598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-1eaf5ba4-36b0-45f5-bb17-ece1135a17b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:47.669598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:47.669598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:47.669598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:47.669598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:47.669598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:47.669598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:47.669598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:47.669598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:47.669598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:47.669598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:47.669598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:47.669598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:48:47.669598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:48:47.669598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:48:47.670671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:48:47.670671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:47.670671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:47.670671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:47.670671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:47.670671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:47.670671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:47.670671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:47.670671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:47.670671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:47.670671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:47.670671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:47.670671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:47.670671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:47.670671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:47.670671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:47.670671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:47.671727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:48:47.671727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:48:47.671727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:48:47.671727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:48:47.671727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:48:47.671727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:48:47.671727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:47.671727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:47.671727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 329/644] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:48:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:47.672206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc08d21-01bb-4fdf-af00-34e241c79f9f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:47.672206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc08d21-01bb-4fdf-af00-34e241c79f9f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:47.672484 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc08d21-01bb-4fdf-af00-34e241c79f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:47.691738 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6bc08d21-01bb-4fdf-af00-34e241c79f9f None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:47.692965 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6bc08d21-01bb-4fdf-af00-34e241c79f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:47.699495 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141babea-865b-454a-ba34-b9c8d3eba666 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:47.735888 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5ec8b-e386-4642-b788-69e9f02a541e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:47.736570 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5ec8b-e386-4642-b788-69e9f02a541e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:47.737053 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5ec8b-e386-4642-b788-69e9f02a541e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:47.737164 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5ec8b-e386-4642-b788-69e9f02a541e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:47.772324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 330/645] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:47.772324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc08d21-01bb-4fdf-af00-34e241c79f9f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:47.818728 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5ec8b-e386-4642-b788-69e9f02a541e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:47.820182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5ec8b-e386-4642-b788-69e9f02a541e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:47.878413 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc08d21-01bb-4fdf-af00-34e241c79f9f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:47.891873 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:47.904404 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:47.916936 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b05fb8-11a5-4425-a0cc-0c7ac5449724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:47.917543 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b05fb8-11a5-4425-a0cc-0c7ac5449724 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:47.917823 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b05fb8-11a5-4425-a0cc-0c7ac5449724 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:47.918095 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b05fb8-11a5-4425-a0cc-0c7ac5449724 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:47.918894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b05fb8-11a5-4425-a0cc-0c7ac5449724 None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:47.919205 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b05fb8-11a5-4425-a0cc-0c7ac5449724 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:47.920608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b05fb8-11a5-4425-a0cc-0c7ac5449724 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:47.922775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 316/646] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:48:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:48:47.926403 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5ec8b-e386-4642-b788-69e9f02a541e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:47.929418 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 331/647] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jun 23 19:48:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:47.934633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b05fb8-11a5-4425-a0cc-0c7ac5449724 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:48.043211 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b05fb8-11a5-4425-a0cc-0c7ac5449724 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:48.047426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5ec8b-e386-4642-b788-69e9f02a541e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:48.079222 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728fe42-4b6b-45f4-a748-40665a182417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:48.080010 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728fe42-4b6b-45f4-a748-40665a182417 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:48.080293 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728fe42-4b6b-45f4-a748-40665a182417 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:48.080598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728fe42-4b6b-45f4-a748-40665a182417 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:48.086402 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c966a2-eee8-4c4c-a44d-1a843614a4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:48.087703 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c966a2-eee8-4c4c-a44d-1a843614a4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:48.088150 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c966a2-eee8-4c4c-a44d-1a843614a4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:48.088461 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c966a2-eee8-4c4c-a44d-1a843614a4d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:48.089265 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c966a2-eee8-4c4c-a44d-1a843614a4d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:48.089578 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c966a2-eee8-4c4c-a44d-1a843614a4d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:48.091015 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c966a2-eee8-4c4c-a44d-1a843614a4d2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:48.092018 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728fe42-4b6b-45f4-a748-40665a182417 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:48.100158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-8728fe42-4b6b-45f4-a748-40665a182417 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:48.100158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:48.100158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:48.100158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:48.100158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:48.100158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:48.100158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:48.100158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:48.100158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:48.100158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:48.100158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:48.100158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:48.100158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:48.100158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:48.100158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:48.100158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:48.101118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:48.101118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:48.101118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:48.101118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:48.101118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:48.101118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:48.101118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:48.101118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:48.101118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:48.101118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:48.101118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:48.101118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:48.101118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:48.101118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:48.101118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:48.101118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:48.101949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:48.101949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:48.101949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:48.101949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:48.101949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:48.101949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:48.101949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:48.101949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:48.101949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:48.101949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:48.101949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:48.101949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:48.101949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 332/648] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jun 23 19:48:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:48.102640 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728fe42-4b6b-45f4-a748-40665a182417 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:48.106480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 317/649] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:48:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:48.107539 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c966a2-eee8-4c4c-a44d-1a843614a4d2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:48.141035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:48.156219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:48.209814 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728fe42-4b6b-45f4-a748-40665a182417 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:48.214374 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c966a2-eee8-4c4c-a44d-1a843614a4d2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:48.251323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd280c9-7d91-40d5-b22a-756419846ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:48.252410 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd280c9-7d91-40d5-b22a-756419846ffc None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:48.252698 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd280c9-7d91-40d5-b22a-756419846ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:48.252974 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd280c9-7d91-40d5-b22a-756419846ffc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:48.256021 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87906c-1f47-441e-bfe2-822c847b8d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:48.256836 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87906c-1f47-441e-bfe2-822c847b8d4a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:48.257117 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87906c-1f47-441e-bfe2-822c847b8d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:48.257390 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87906c-1f47-441e-bfe2-822c847b8d4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:48.265378 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd280c9-7d91-40d5-b22a-756419846ffc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:48.267591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd280c9-7d91-40d5-b22a-756419846ffc None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:48.267965 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87906c-1f47-441e-bfe2-822c847b8d4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:48.269443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87906c-1f47-441e-bfe2-822c847b8d4a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:48.287206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-7bd280c9-7d91-40d5-b22a-756419846ffc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:48.287206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:48.287206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:48.287206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:48.287206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:48.287206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:48.287206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:48.287206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:48.287206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:48.287206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:48.287206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:48.287206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:48.287206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:48.287206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:48.287206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:48.288130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:48.288130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:48.288130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:48.288130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:48.288130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:48.288130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:48.288130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:48.288130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:48.288130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:48.288130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:48.288130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:48.288130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:48.288130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:48.288130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:48.288130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:48.288130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:48.288130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:48.289442 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:48.289442 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:48.289442 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:48.289442 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:48.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-da87906c-1f47-441e-bfe2-822c847b8d4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:48.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:48.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:48.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:48.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:48.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:48.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:48.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:48.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:48.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:48.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:48.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:48.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:48.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:48.291443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:48.292285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:48.292285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:48.292285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:48.292285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:48.292285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:48.292285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:48.292285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:48.292285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:48.292285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:48.292285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:48.292285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:48.292285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:48.292285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:48.292285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:48.292285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:48.292285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:48.292285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:48.295033 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:48.295033 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:48.295033 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:48.295033 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:48.295539 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 333/650] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:48.295539 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd280c9-7d91-40d5-b22a-756419846ffc None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:48.296751 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 318/651] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:48.298674 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87906c-1f47-441e-bfe2-822c847b8d4a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:48.392260 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:48.398827 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:48.403007 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd280c9-7d91-40d5-b22a-756419846ffc None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:48.409338 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87906c-1f47-441e-bfe2-822c847b8d4a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:48.438392 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce78b77-9d11-4e15-8514-c00c63333b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:48.439064 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce78b77-9d11-4e15-8514-c00c63333b3a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:48.439357 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce78b77-9d11-4e15-8514-c00c63333b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:48.439688 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce78b77-9d11-4e15-8514-c00c63333b3a None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:48.440394 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce78b77-9d11-4e15-8514-c00c63333b3a None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:48.440705 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce78b77-9d11-4e15-8514-c00c63333b3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:48.442091 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce78b77-9d11-4e15-8514-c00c63333b3a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:48.444939 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbfe010-d0be-4a8a-a955-7491084dfe20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:48.445853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbfe010-d0be-4a8a-a955-7491084dfe20 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:48.446154 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbfe010-d0be-4a8a-a955-7491084dfe20 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:48.446412 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbfe010-d0be-4a8a-a955-7491084dfe20 None admin] PATH_INFO: `/v3/projects/155180ba63c446ca889a6eb3f94cc31a/users/96db03079816461bb322633f64b9fb9d/roles/5f27c72acb19431fa3223c962bc42938` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:48.456862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 334/652] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jun 23 19:48:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:48.456862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce78b77-9d11-4e15-8514-c00c63333b3a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:48.463527 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbfe010-d0be-4a8a-a955-7491084dfe20 None admin] RBAC: Authorizing `identity:create_grant(project_id=155180ba63c446ca889a6eb3f94cc31a, user_id=96db03079816461bb322633f64b9fb9d, role_id=5f27c72acb19431fa3223c962bc42938)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:48.464969 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbfe010-d0be-4a8a-a955-7491084dfe20 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:48.528030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 319/653] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jun 23 19:48:48 2022] PUT /identity/v3/projects/155180ba63c446ca889a6eb3f94cc31a/users/96db03079816461bb322633f64b9fb9d/roles/5f27c72acb19431fa3223c962bc42938 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:48:48.529298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbfe010-d0be-4a8a-a955-7491084dfe20 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:48.568538 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce78b77-9d11-4e15-8514-c00c63333b3a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:48.608909 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff38ebb-1278-44e2-91ce-b6a0643d8eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:48.610015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff38ebb-1278-44e2-91ce-b6a0643d8eec None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:48.610530 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff38ebb-1278-44e2-91ce-b6a0643d8eec None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:48.611007 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff38ebb-1278-44e2-91ce-b6a0643d8eec None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:48.611949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff38ebb-1278-44e2-91ce-b6a0643d8eec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:48.612492 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff38ebb-1278-44e2-91ce-b6a0643d8eec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:48.614144 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff38ebb-1278-44e2-91ce-b6a0643d8eec None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:48.628803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 335/654] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:48.630675 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff38ebb-1278-44e2-91ce-b6a0643d8eec None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:48.657007 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:48.664708 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbfe010-d0be-4a8a-a955-7491084dfe20 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:48.711569 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85a103-84d3-4d77-a50c-13f6a062c906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:48.712246 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85a103-84d3-4d77-a50c-13f6a062c906 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:48.712557 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85a103-84d3-4d77-a50c-13f6a062c906 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:48.712837 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85a103-84d3-4d77-a50c-13f6a062c906 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:48.713623 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85a103-84d3-4d77-a50c-13f6a062c906 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:48.713881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85a103-84d3-4d77-a50c-13f6a062c906 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:48.715311 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85a103-84d3-4d77-a50c-13f6a062c906 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:48.717257 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:48.731568 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 320/655] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:48.732474 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85a103-84d3-4d77-a50c-13f6a062c906 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:48.753021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff38ebb-1278-44e2-91ce-b6a0643d8eec None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:48.797117 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc21c8d-ed0c-4723-869d-dde7a111e443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:48.797906 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc21c8d-ed0c-4723-869d-dde7a111e443 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:48.798181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc21c8d-ed0c-4723-869d-dde7a111e443 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:48.798449 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc21c8d-ed0c-4723-869d-dde7a111e443 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:48.809451 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc21c8d-ed0c-4723-869d-dde7a111e443 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:48.810963 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc21c8d-ed0c-4723-869d-dde7a111e443 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:48.830200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-1fc21c8d-ed0c-4723-869d-dde7a111e443 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:48.830200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:48.830200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:48.830200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:48.830200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:48.830200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:48.830200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:48.830200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:48.830200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:48.830200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:48.830200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:48.830200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:48.830200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:48.830200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:48.830200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:48.831125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:48.831125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:48.831125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:48.831125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:48.831125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:48.831125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:48.831125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:48.831125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:48.831125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:48.831125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:48.831125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:48.831125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:48.831125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:48.831125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:48.831125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:48.831125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:48.831125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:48.832017 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:48.832017 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:48.832017 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:48.832017 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:48.832017 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 336/656] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:48.836627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc21c8d-ed0c-4723-869d-dde7a111e443 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:48.854348 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85a103-84d3-4d77-a50c-13f6a062c906 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:48.899938 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e7ddc-d07a-4be6-bd9d-f5e265df987d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:48.901002 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e7ddc-d07a-4be6-bd9d-f5e265df987d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:48.901673 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e7ddc-d07a-4be6-bd9d-f5e265df987d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:48.901963 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e7ddc-d07a-4be6-bd9d-f5e265df987d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:48.911229 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e7ddc-d07a-4be6-bd9d-f5e265df987d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:48.913626 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:48.915051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e7ddc-d07a-4be6-bd9d-f5e265df987d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:48.933259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-313e7ddc-d07a-4be6-bd9d-f5e265df987d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:48.933259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:48.933259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:48.933259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:48.933259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:48.933259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:48.933259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:48.933259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:48.933259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:48.933259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:48.933259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:48.933259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:48.933259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:48:48.933259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:48:48.933259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:48:48.934398 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:48:48.934398 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:48.934398 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:48.934398 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:48.934398 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:48.934398 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:48.934398 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:48.934398 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:48.934398 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:48.934398 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:48.934398 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:48.934398 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:48.934398 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:48.934398 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:48.934398 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:48.934398 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:48.934398 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:48.935466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:48:48.935466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:48:48.935466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:48:48.935466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:48:48.935466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:48:48.935466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:48:48.935466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:48.935466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:48.938664 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 321/657] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:48:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:48.939742 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e7ddc-d07a-4be6-bd9d-f5e265df987d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:48.951659 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc21c8d-ed0c-4723-869d-dde7a111e443 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:48.996347 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8552c3a-b0eb-4b70-81bc-6618bf3ccb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:48.997665 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8552c3a-b0eb-4b70-81bc-6618bf3ccb57 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:48.998625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8552c3a-b0eb-4b70-81bc-6618bf3ccb57 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:48.999190 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8552c3a-b0eb-4b70-81bc-6618bf3ccb57 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:49.008257 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:49.011995 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8552c3a-b0eb-4b70-81bc-6618bf3ccb57 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:49.014268 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8552c3a-b0eb-4b70-81bc-6618bf3ccb57 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:49.033608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-c8552c3a-b0eb-4b70-81bc-6618bf3ccb57 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 19:48:49.033608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:49.033608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:49.033608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:49.033608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:49.033608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:49.033608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:49.033608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:49.033608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:49.033608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:49.033608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:49.033608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:49.033608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:48:49.033608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:48:49.033608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:48:49.033608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:48:49.034798 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:49.034798 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:49.034798 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:48:49.034798 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:49.034798 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:48:49.034798 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:49.034798 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:49.034798 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:49.034798 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:49.034798 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:49.034798 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:49.034798 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:49.034798 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:49.034798 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:49.034798 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:49.034798 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:49.034798 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:49.034798 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:49.035917 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:49.035917 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:49.035917 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:48:49.035917 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:48:49.035917 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:48:49.035917 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:48:49.035917 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:48:49.035917 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:48:49.035917 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 19:48:49.035917 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:49.043141 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 337/658] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:49.044286 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8552c3a-b0eb-4b70-81bc-6618bf3ccb57 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:49.063847 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e7ddc-d07a-4be6-bd9d-f5e265df987d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:49.101345 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff02376e-0456-45aa-9691-94368a1c0379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:49.102124 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02376e-0456-45aa-9691-94368a1c0379 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:49.102311 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02376e-0456-45aa-9691-94368a1c0379 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:49.102538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02376e-0456-45aa-9691-94368a1c0379 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:49.103493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff02376e-0456-45aa-9691-94368a1c0379 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:49.103801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff02376e-0456-45aa-9691-94368a1c0379 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:49.105408 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff02376e-0456-45aa-9691-94368a1c0379 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:49.117695 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 322/659] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:49.118386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff02376e-0456-45aa-9691-94368a1c0379 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:49.173097 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8552c3a-b0eb-4b70-81bc-6618bf3ccb57 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:49.204740 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:49.212176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658cd910-4e94-4615-aa7c-fce163be132b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:49.213558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cd910-4e94-4615-aa7c-fce163be132b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:49.217886 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cd910-4e94-4615-aa7c-fce163be132b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:49.217886 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cd910-4e94-4615-aa7c-fce163be132b None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:49.217886 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658cd910-4e94-4615-aa7c-fce163be132b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:49.217886 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658cd910-4e94-4615-aa7c-fce163be132b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:49.217886 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658cd910-4e94-4615-aa7c-fce163be132b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:49.238838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 338/660] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:48:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:49.238838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658cd910-4e94-4615-aa7c-fce163be132b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:49.259165 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff02376e-0456-45aa-9691-94368a1c0379 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:49.275047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:49.303177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72f801-0679-4033-90b8-34f24a0687c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:49.303859 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72f801-0679-4033-90b8-34f24a0687c6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:49.304149 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72f801-0679-4033-90b8-34f24a0687c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:49.304447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72f801-0679-4033-90b8-34f24a0687c6 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:49.305592 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72f801-0679-4033-90b8-34f24a0687c6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:49.305902 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72f801-0679-4033-90b8-34f24a0687c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:49.308000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a72f801-0679-4033-90b8-34f24a0687c6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:49.337850 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 323/661] 10.222.0.44 () {56 vars in 1137 bytes} [Thu Jun 23 19:48:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:49.338932 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72f801-0679-4033-90b8-34f24a0687c6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:49.355166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658cd910-4e94-4615-aa7c-fce163be132b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:49.392548 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34fbbe-5c7e-4f67-b87a-c3397ce4ac9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:49.393718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34fbbe-5c7e-4f67-b87a-c3397ce4ac9e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:49.394196 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34fbbe-5c7e-4f67-b87a-c3397ce4ac9e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:49.394663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34fbbe-5c7e-4f67-b87a-c3397ce4ac9e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:49.404563 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d34fbbe-5c7e-4f67-b87a-c3397ce4ac9e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:49.406162 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d34fbbe-5c7e-4f67-b87a-c3397ce4ac9e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:49.421181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-7d34fbbe-5c7e-4f67-b87a-c3397ce4ac9e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 19:48:49.421181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:49.421181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:49.421181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:49.421181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:49.421181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:49.421181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:49.421181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:49.421181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:49.421181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:49.421181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:49.421181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:49.421181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:48:49.421181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:48:49.421181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:48:49.421181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:48:49.422334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:49.422334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:49.422334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:48:49.422334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:49.422334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:48:49.422334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:49.422334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:49.422334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:49.422334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:49.422334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:49.422334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:49.422334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:49.422334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:49.422334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:49.422334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:49.422334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:49.422334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:49.422334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:49.423284 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:49.423284 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:49.423284 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:48:49.423284 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:48:49.423284 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:48:49.423284 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:48:49.423284 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:48:49.423284 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:48:49.423284 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 19:48:49.423284 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:49.423284 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 339/662] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jun 23 19:48:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:49.423284 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34fbbe-5c7e-4f67-b87a-c3397ce4ac9e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:49.452146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72f801-0679-4033-90b8-34f24a0687c6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:49.486116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8168f155-84f5-428d-af46-03f8678e6a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:49.486758 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8168f155-84f5-428d-af46-03f8678e6a6a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:49.487036 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8168f155-84f5-428d-af46-03f8678e6a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:49.487303 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8168f155-84f5-428d-af46-03f8678e6a6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:49.488131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8168f155-84f5-428d-af46-03f8678e6a6a None admin] RBAC: Adding query filter params (role.id=5f27c72acb19431fa3223c962bc42938, scope.project.id=155180ba63c446ca889a6eb3f94cc31a, user.id=1dd72f952df34ad8853fe84673a41f45) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:49.488455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8168f155-84f5-428d-af46-03f8678e6a6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:49.489058 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:49.489942 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8168f155-84f5-428d-af46-03f8678e6a6a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:49.497717 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 324/663] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:48:49 2022] GET /identity/v3/role_assignments?user.id=1dd72f952df34ad8853fe84673a41f45&scope.project.id=155180ba63c446ca889a6eb3f94cc31a&role.id=5f27c72acb19431fa3223c962bc42938 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:49.500126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb74b3-fd20-45f1-a378-2a2e1864a775 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:49.507434 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb74b3-fd20-45f1-a378-2a2e1864a775 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:49.507434 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb74b3-fd20-45f1-a378-2a2e1864a775 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:49.507434 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 325/664] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:49.507434 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb74b3-fd20-45f1-a378-2a2e1864a775 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:49.548024 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:49.548501 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34fbbe-5c7e-4f67-b87a-c3397ce4ac9e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:49.592180 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08bbeea-e96e-426d-a351-5497af092526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:49.593315 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08bbeea-e96e-426d-a351-5497af092526 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:49.593874 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08bbeea-e96e-426d-a351-5497af092526 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:49.594420 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08bbeea-e96e-426d-a351-5497af092526 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:49.635789 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb74b3-fd20-45f1-a378-2a2e1864a775 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:49.678605 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76cb63-77b2-4913-806e-cf17cf6328c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:49.679866 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76cb63-77b2-4913-806e-cf17cf6328c9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:49.680545 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76cb63-77b2-4913-806e-cf17cf6328c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:49.681346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76cb63-77b2-4913-806e-cf17cf6328c9 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:49.682212 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76cb63-77b2-4913-806e-cf17cf6328c9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:49.682529 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76cb63-77b2-4913-806e-cf17cf6328c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:49.683993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76cb63-77b2-4913-806e-cf17cf6328c9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:49.689562 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08bbeea-e96e-426d-a351-5497af092526 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:49.691068 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08bbeea-e96e-426d-a351-5497af092526 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:49.709429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 326/665] 10.222.0.44 () {56 vars in 1139 bytes} [Thu Jun 23 19:48:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:49.710753 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d9628-ae6a-473c-bed6-52821d21c962 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:49.711044 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d9628-ae6a-473c-bed6-52821d21c962 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:49.711323 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d9628-ae6a-473c-bed6-52821d21c962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:49.730693 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-1a4d9628-ae6a-473c-bed6-52821d21c962 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:49.732224 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1a4d9628-ae6a-473c-bed6-52821d21c962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:49.761345 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:49.818212 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 327/666] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:49.818503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d9628-ae6a-473c-bed6-52821d21c962 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:49.832023 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 340/667] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:48:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:48:49.838012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08bbeea-e96e-426d-a351-5497af092526 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:49.856460 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:49.947521 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d9628-ae6a-473c-bed6-52821d21c962 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:49.965340 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08bbeea-e96e-426d-a351-5497af092526 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:49.984005 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af810461-e5b6-48fb-8ef7-4aa8c3fd19a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:49.985008 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af810461-e5b6-48fb-8ef7-4aa8c3fd19a1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:49.985008 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af810461-e5b6-48fb-8ef7-4aa8c3fd19a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:49.985225 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af810461-e5b6-48fb-8ef7-4aa8c3fd19a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:50.014760 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0860dfe-976e-49fa-8d00-ac944472fa63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:50.016069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0860dfe-976e-49fa-8d00-ac944472fa63 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:50.016919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0860dfe-976e-49fa-8d00-ac944472fa63 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:50.017348 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0860dfe-976e-49fa-8d00-ac944472fa63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:50.036123 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0860dfe-976e-49fa-8d00-ac944472fa63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:50.038873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0860dfe-976e-49fa-8d00-ac944472fa63 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:50.063927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-b0860dfe-976e-49fa-8d00-ac944472fa63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:50.063927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:50.063927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:50.063927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:50.063927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:50.063927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:50.063927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:50.063927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:50.063927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:50.063927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:50.063927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:50.063927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:50.063927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:50.063927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:50.063927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:50.064958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:50.064958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:50.064958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:50.064958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:50.064958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:50.064958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:50.064958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:50.064958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:50.064958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:50.064958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:50.064958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:50.064958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:50.064958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:50.064958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:50.064958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:50.064958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:50.064958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:50.065835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:50.065835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:50.065835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:50.065835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:50.068112 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 341/668] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:48:50.068730 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af810461-e5b6-48fb-8ef7-4aa8c3fd19a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:50.069429 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0860dfe-976e-49fa-8d00-ac944472fa63 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:50.070402 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af810461-e5b6-48fb-8ef7-4aa8c3fd19a1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:50.073554 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:50.096207 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:50.177980 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0860dfe-976e-49fa-8d00-ac944472fa63 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:50.181291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 328/669] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:48:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:48:50.181739 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af810461-e5b6-48fb-8ef7-4aa8c3fd19a1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:50.218483 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c93541-afb2-4afd-967d-f12488d15fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:50.219288 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c93541-afb2-4afd-967d-f12488d15fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:50.219578 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c93541-afb2-4afd-967d-f12488d15fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:50.219850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c93541-afb2-4afd-967d-f12488d15fa0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:50.241100 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c93541-afb2-4afd-967d-f12488d15fa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:50.251869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-58c93541-afb2-4afd-967d-f12488d15fa0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:48:50.251869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:50.251869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:50.251869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:50.251869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:50.251869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:50.251869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:50.251869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:50.251869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:50.251869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:50.251869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:50.251869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:50.251869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:50.251869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:50.251869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:50.251869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:50.252855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:50.252855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:50.252855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:50.252855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:50.252855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:50.252855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:50.252855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:50.252855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:50.252855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:50.252855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:50.252855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:50.252855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:50.252855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:50.252855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:50.252855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:50.252855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:50.253695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:50.253695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:50.253695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:50.253695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:50.253695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:50.253695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:50.253695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:50.253695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:50.253695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:50.253695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:50.253695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:48:50.253695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:50.258355 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 342/670] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:48:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:50.259015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c93541-afb2-4afd-967d-f12488d15fa0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:50.314247 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af810461-e5b6-48fb-8ef7-4aa8c3fd19a1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:50.333510 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:50.346185 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:50.360676 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36544f13-d7b9-414a-af1c-8f7bfd7ae148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:50.362203 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36544f13-d7b9-414a-af1c-8f7bfd7ae148 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:50.362697 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36544f13-d7b9-414a-af1c-8f7bfd7ae148 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:50.363189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36544f13-d7b9-414a-af1c-8f7bfd7ae148 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:50.364124 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36544f13-d7b9-414a-af1c-8f7bfd7ae148 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:50.365141 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36544f13-d7b9-414a-af1c-8f7bfd7ae148 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:50.367389 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36544f13-d7b9-414a-af1c-8f7bfd7ae148 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:50.379185 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c93541-afb2-4afd-967d-f12488d15fa0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:50.383901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 329/671] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:50.384995 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36544f13-d7b9-414a-af1c-8f7bfd7ae148 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:50.429333 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f052ebf-a45a-489c-bad3-eb1252704a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:50.430528 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f052ebf-a45a-489c-bad3-eb1252704a44 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:50.431018 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f052ebf-a45a-489c-bad3-eb1252704a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:50.431497 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f052ebf-a45a-489c-bad3-eb1252704a44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:50.443864 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f052ebf-a45a-489c-bad3-eb1252704a44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:50.446737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f052ebf-a45a-489c-bad3-eb1252704a44 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:50.463804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-4f052ebf-a45a-489c-bad3-eb1252704a44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:50.463804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:50.463804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:50.463804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:50.463804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:50.463804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:50.463804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:50.463804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:50.463804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:50.463804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:50.463804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:50.463804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:50.463804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:50.463804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:50.463804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:50.464721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:50.464721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:50.464721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:50.464721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:50.464721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:50.464721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:50.464721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:50.464721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:50.464721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:50.464721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:50.464721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:50.464721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:50.464721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:50.464721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:50.464721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:50.464721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:50.464721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:50.465722 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:50.465722 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:50.465722 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:50.465722 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:50.468053 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 343/672] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:50.469063 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f052ebf-a45a-489c-bad3-eb1252704a44 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:50.502720 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36544f13-d7b9-414a-af1c-8f7bfd7ae148 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:50.539063 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435c5b7-c19a-4d05-839b-5fb59cab9669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:50.539840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435c5b7-c19a-4d05-839b-5fb59cab9669 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:50.540051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435c5b7-c19a-4d05-839b-5fb59cab9669 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:50.540284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e435c5b7-c19a-4d05-839b-5fb59cab9669 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:50.541069 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435c5b7-c19a-4d05-839b-5fb59cab9669 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:50.541377 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435c5b7-c19a-4d05-839b-5fb59cab9669 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:50.542763 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e435c5b7-c19a-4d05-839b-5fb59cab9669 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:50.555847 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 330/673] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:50.556773 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435c5b7-c19a-4d05-839b-5fb59cab9669 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:50.579153 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f052ebf-a45a-489c-bad3-eb1252704a44 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:50.601541 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:50.611015 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:50.629710 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f22aad-a3cc-4990-bccc-dd9cdfdc85b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:50.632993 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f22aad-a3cc-4990-bccc-dd9cdfdc85b5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:50.633768 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f22aad-a3cc-4990-bccc-dd9cdfdc85b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:50.634466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f22aad-a3cc-4990-bccc-dd9cdfdc85b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:50.644075 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f22aad-a3cc-4990-bccc-dd9cdfdc85b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:50.646261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f22aad-a3cc-4990-bccc-dd9cdfdc85b5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:50.666637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-f5f22aad-a3cc-4990-bccc-dd9cdfdc85b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:50.666637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:50.666637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:50.666637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:50.666637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:50.666637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:50.666637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:50.666637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:50.666637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:50.666637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:50.666637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:50.666637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:50.666637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:48:50.666637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:48:50.666637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:48:50.667561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:48:50.667561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:50.667561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:50.667561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:50.667561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:50.667561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:50.667561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:50.667561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:50.667561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:50.667561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:50.667561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:50.667561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:50.667561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:50.667561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:50.667561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:50.667561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:50.667561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:50.669047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:48:50.669047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:48:50.669047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:48:50.669047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:48:50.669047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:48:50.669047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:48:50.669047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:50.669047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:50.669047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 344/674] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:48:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:50.669047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f22aad-a3cc-4990-bccc-dd9cdfdc85b5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:50.672252 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e435c5b7-c19a-4d05-839b-5fb59cab9669 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:50.709655 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4ee8e6-3e3e-4018-af86-02fd754c434a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:50.710320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ee8e6-3e3e-4018-af86-02fd754c434a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:50.710603 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ee8e6-3e3e-4018-af86-02fd754c434a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:50.710880 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4ee8e6-3e3e-4018-af86-02fd754c434a None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:50.711601 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ee8e6-3e3e-4018-af86-02fd754c434a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:50.711909 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ee8e6-3e3e-4018-af86-02fd754c434a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:50.713336 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4ee8e6-3e3e-4018-af86-02fd754c434a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:50.725224 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 331/675] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:50.726488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e388b5-a63a-4ac1-bc8e-9fbc3d0acdb0 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:50.726772 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e388b5-a63a-4ac1-bc8e-9fbc3d0acdb0 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:50.727045 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e388b5-a63a-4ac1-bc8e-9fbc3d0acdb0 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:50.732275 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 332/676] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:50.732275 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e388b5-a63a-4ac1-bc8e-9fbc3d0acdb0 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:50.795573 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f22aad-a3cc-4990-bccc-dd9cdfdc85b5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:50.833146 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb20982-6440-4733-ae9f-1888f7b1a784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:50.833935 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb20982-6440-4733-ae9f-1888f7b1a784 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:50.834215 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb20982-6440-4733-ae9f-1888f7b1a784 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:50.834485 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb20982-6440-4733-ae9f-1888f7b1a784 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:50.840211 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e388b5-a63a-4ac1-bc8e-9fbc3d0acdb0 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:50.844867 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb20982-6440-4733-ae9f-1888f7b1a784 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:50.846349 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb20982-6440-4733-ae9f-1888f7b1a784 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:50.850244 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:50.863975 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-6eb20982-6440-4733-ae9f-1888f7b1a784 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:50.863975 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:50.863975 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:50.863975 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:50.863975 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:50.863975 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:50.863975 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:50.863975 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:50.863975 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:50.863975 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:50.863975 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:50.863975 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:50.863975 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:50.863975 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:50.863975 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:50.865637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:50.865637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:50.865637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:50.865637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:50.865637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:50.865637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:50.865637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:50.865637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:50.865637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:50.865637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:50.865637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:50.865637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:50.865637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:50.865637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:50.865637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:50.865637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:50.865637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:50.866748 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:50.866748 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:50.866748 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:50.866748 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:50.870251 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 345/677] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:50.871059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb20982-6440-4733-ae9f-1888f7b1a784 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:50.880586 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a7eeb-6e98-4bf1-af61-3e9fb82517cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:50.881363 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a7eeb-6e98-4bf1-af61-3e9fb82517cf None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:50.881753 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a7eeb-6e98-4bf1-af61-3e9fb82517cf None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:50.882104 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a7eeb-6e98-4bf1-af61-3e9fb82517cf None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:50.882971 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a7eeb-6e98-4bf1-af61-3e9fb82517cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:50.883363 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a7eeb-6e98-4bf1-af61-3e9fb82517cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:50.884917 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a7eeb-6e98-4bf1-af61-3e9fb82517cf None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:50.897333 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 333/678] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:48:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:50.898617 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8631b37-2d3d-4818-b5a3-9f5838c9ff15 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:50.899119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8631b37-2d3d-4818-b5a3-9f5838c9ff15 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:50.899474 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8631b37-2d3d-4818-b5a3-9f5838c9ff15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:50.905871 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:50.915883 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f8631b37-2d3d-4818-b5a3-9f5838c9ff15 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:50.918991 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f8631b37-2d3d-4818-b5a3-9f5838c9ff15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:50.983180 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb20982-6440-4733-ae9f-1888f7b1a784 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:50.986669 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 334/679] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:50.988243 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8631b37-2d3d-4818-b5a3-9f5838c9ff15 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:51.035287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1bc05-6817-4ebf-b65b-50ec4171d7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:51.036661 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1bc05-6817-4ebf-b65b-50ec4171d7fb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:51.036949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1bc05-6817-4ebf-b65b-50ec4171d7fb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:51.037222 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1bc05-6817-4ebf-b65b-50ec4171d7fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:51.044164 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae1bc05-6817-4ebf-b65b-50ec4171d7fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:51.045651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae1bc05-6817-4ebf-b65b-50ec4171d7fb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:51.059941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-dae1bc05-6817-4ebf-b65b-50ec4171d7fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:51.059941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:51.059941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:51.059941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:51.059941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:51.059941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:51.059941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:51.059941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:51.059941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:51.059941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:51.059941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:51.059941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:51.059941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:48:51.059941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:48:51.059941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:48:51.061012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:48:51.061012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:51.061012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:51.061012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:51.061012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:51.061012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:51.061012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:51.061012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:51.061012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:51.061012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:51.061012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:51.061012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:51.061012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:51.061012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:51.061012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:51.061012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:51.061012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:51.061926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:48:51.061926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:48:51.061926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:48:51.061926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:48:51.061926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:48:51.061926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:48:51.061926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:51.061926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:51.061926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 346/680] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:48:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:51.062410 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1bc05-6817-4ebf-b65b-50ec4171d7fb None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:51.106380 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:51.113348 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8631b37-2d3d-4818-b5a3-9f5838c9ff15 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:51.152119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861192eb-37a2-4a92-add3-e08ec22c692f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:51.152736 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861192eb-37a2-4a92-add3-e08ec22c692f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:51.153021 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861192eb-37a2-4a92-add3-e08ec22c692f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:51.153300 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861192eb-37a2-4a92-add3-e08ec22c692f None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:51.154023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861192eb-37a2-4a92-add3-e08ec22c692f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:51.154325 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861192eb-37a2-4a92-add3-e08ec22c692f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:51.155712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861192eb-37a2-4a92-add3-e08ec22c692f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:51.158296 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:51.171966 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 335/681] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:51.171966 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861192eb-37a2-4a92-add3-e08ec22c692f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:51.175561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1bc05-6817-4ebf-b65b-50ec4171d7fb None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:51.211079 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d63e1c-7865-4b13-966f-e71bacca985a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:51.211738 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d63e1c-7865-4b13-966f-e71bacca985a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:51.212020 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d63e1c-7865-4b13-966f-e71bacca985a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:51.212741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d63e1c-7865-4b13-966f-e71bacca985a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:51.213266 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d63e1c-7865-4b13-966f-e71bacca985a None admin] RBAC: Adding query filter params (role.id=5f27c72acb19431fa3223c962bc42938, scope.project.id=155180ba63c446ca889a6eb3f94cc31a, user.id=21ae3b8971b44d928e9f6c0539858636) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:51.213579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d63e1c-7865-4b13-966f-e71bacca985a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:51.215130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d63e1c-7865-4b13-966f-e71bacca985a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:51.228770 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 347/682] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:48:51 2022] GET /identity/v3/role_assignments?user.id=21ae3b8971b44d928e9f6c0539858636&scope.project.id=155180ba63c446ca889a6eb3f94cc31a&role.id=5f27c72acb19431fa3223c962bc42938 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:51.228770 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d63e1c-7865-4b13-966f-e71bacca985a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:51.295822 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861192eb-37a2-4a92-add3-e08ec22c692f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:51.337307 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a212b-e9aa-44eb-b8ae-23907f000931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:51.338246 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a212b-e9aa-44eb-b8ae-23907f000931 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:51.338685 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a212b-e9aa-44eb-b8ae-23907f000931 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:51.339044 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a212b-e9aa-44eb-b8ae-23907f000931 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:51.351919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a212b-e9aa-44eb-b8ae-23907f000931 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:51.360790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-750a212b-e9aa-44eb-b8ae-23907f000931 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:51.360790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:51.360790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:51.360790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:51.360790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:51.360790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:51.360790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:51.360790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:51.360790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:51.360790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:51.360790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:51.360790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:51.360790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:51.360790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:51.360790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:51.360790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:51.361893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:51.361893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:51.361893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:51.361893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:51.361893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:51.361893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:51.361893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:51.361893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:51.361893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:51.361893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:51.361893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:51.361893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:51.361893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:51.361893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:51.361893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:51.361893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:51.362912 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:51.362912 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:51.362912 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:51.362912 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:51.362912 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:51.362912 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:51.362912 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:51.362912 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:51.362912 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:51.362912 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:51.362912 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:51.362912 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:51.365946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 336/683] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jun 23 19:48:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:51.365946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a212b-e9aa-44eb-b8ae-23907f000931 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:51.366785 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d63e1c-7865-4b13-966f-e71bacca985a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:51.392709 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:51.417407 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e57ed-9dcd-4e32-858b-6de150717bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:51.418436 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e57ed-9dcd-4e32-858b-6de150717bbe None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:51.418954 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e57ed-9dcd-4e32-858b-6de150717bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:51.419434 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e57ed-9dcd-4e32-858b-6de150717bbe None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:51.420455 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e57ed-9dcd-4e32-858b-6de150717bbe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:51.420992 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e57ed-9dcd-4e32-858b-6de150717bbe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:51.422638 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e57ed-9dcd-4e32-858b-6de150717bbe None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:51.436141 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 348/684] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:48:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:51.437292 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e57ed-9dcd-4e32-858b-6de150717bbe None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:51.452241 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:51.496438 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a212b-e9aa-44eb-b8ae-23907f000931 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:51.545135 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb40d1-eb36-41b8-8f09-2f728089a15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:51.546180 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb40d1-eb36-41b8-8f09-2f728089a15d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:51.546467 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb40d1-eb36-41b8-8f09-2f728089a15d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:51.546991 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb40d1-eb36-41b8-8f09-2f728089a15d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:51.554230 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e57ed-9dcd-4e32-858b-6de150717bbe None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:51.558322 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb40d1-eb36-41b8-8f09-2f728089a15d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:51.559790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb40d1-eb36-41b8-8f09-2f728089a15d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:51.579611 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-1bcb40d1-eb36-41b8-8f09-2f728089a15d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 19:48:51.579611 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:51.579611 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:51.579611 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:51.579611 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:51.579611 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:51.579611 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:51.579611 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:51.579611 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:51.579611 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:51.579611 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:51.579611 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:51.579611 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:48:51.579611 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:48:51.579611 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:48:51.579611 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:48:51.580658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:51.580658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:51.580658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:48:51.580658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:51.580658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:48:51.580658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:51.580658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:51.580658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:51.580658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:51.580658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:51.580658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:51.580658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:51.580658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:51.580658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:51.580658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:51.580658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:51.580658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:51.580658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:51.581604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:51.581604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:51.581604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:48:51.581604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:48:51.581604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:48:51.581604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:48:51.581604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:48:51.581604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:48:51.581604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 19:48:51.581604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:51.582150 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 337/685] 10.222.0.44 () {56 vars in 1093 bytes} [Thu Jun 23 19:48:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:51.584655 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb40d1-eb36-41b8-8f09-2f728089a15d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:51.595200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a268e-c6d0-4365-a3df-e0e4a872a682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:51.595869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a268e-c6d0-4365-a3df-e0e4a872a682 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:51.596146 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a268e-c6d0-4365-a3df-e0e4a872a682 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:51.596441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a268e-c6d0-4365-a3df-e0e4a872a682 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:51.597305 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3a268e-c6d0-4365-a3df-e0e4a872a682 None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:51.597608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3a268e-c6d0-4365-a3df-e0e4a872a682 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:51.599052 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3a268e-c6d0-4365-a3df-e0e4a872a682 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:51.608029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 349/686] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jun 23 19:48:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:51.613230 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a268e-c6d0-4365-a3df-e0e4a872a682 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:51.689412 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:51.694847 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:51.721746 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb40d1-eb36-41b8-8f09-2f728089a15d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:51.721945 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a268e-c6d0-4365-a3df-e0e4a872a682 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:51.758663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c7b2d-7641-41c3-a20b-eb1321279a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:51.758900 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f524902-ac25-44ca-bcc3-bfa9e8113c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:51.759431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c7b2d-7641-41c3-a20b-eb1321279a9c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:51.759531 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f524902-ac25-44ca-bcc3-bfa9e8113c55 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:51.759716 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c7b2d-7641-41c3-a20b-eb1321279a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:51.759945 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f524902-ac25-44ca-bcc3-bfa9e8113c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:51.760048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c7b2d-7641-41c3-a20b-eb1321279a9c None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:51.760290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f524902-ac25-44ca-bcc3-bfa9e8113c55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:51.760843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21c7b2d-7641-41c3-a20b-eb1321279a9c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:51.761150 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21c7b2d-7641-41c3-a20b-eb1321279a9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:51.761294 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f524902-ac25-44ca-bcc3-bfa9e8113c55 None admin] RBAC: Adding query filter params (role.id=5f27c72acb19431fa3223c962bc42938, scope.project.id=155180ba63c446ca889a6eb3f94cc31a, user.id=332fbbefba114da4a01fdd496fff1e31) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:51.761700 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f524902-ac25-44ca-bcc3-bfa9e8113c55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:51.762580 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21c7b2d-7641-41c3-a20b-eb1321279a9c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:51.763263 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f524902-ac25-44ca-bcc3-bfa9e8113c55 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:51.780592 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 338/687] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:48:51 2022] GET /identity/v3/role_assignments?user.id=332fbbefba114da4a01fdd496fff1e31&scope.project.id=155180ba63c446ca889a6eb3f94cc31a&role.id=5f27c72acb19431fa3223c962bc42938 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:51.780592 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f524902-ac25-44ca-bcc3-bfa9e8113c55 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:51.797965 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 350/688] 10.222.0.44 () {56 vars in 1133 bytes} [Thu Jun 23 19:48:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:51.799609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031dbda-09e7-4ce8-89c2-2fb0609c7e94 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:51.799977 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031dbda-09e7-4ce8-89c2-2fb0609c7e94 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:51.800348 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031dbda-09e7-4ce8-89c2-2fb0609c7e94 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:51.801991 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 351/689] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:51.802665 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031dbda-09e7-4ce8-89c2-2fb0609c7e94 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:51.903988 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f524902-ac25-44ca-bcc3-bfa9e8113c55 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:51.949768 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031dbda-09e7-4ce8-89c2-2fb0609c7e94 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:51.951670 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9523e-4e96-44f8-8117-600c60c24ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:51.952614 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9523e-4e96-44f8-8117-600c60c24ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:51.952916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9523e-4e96-44f8-8117-600c60c24ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:51.953725 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9523e-4e96-44f8-8117-600c60c24ed4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:51.962918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:51.965342 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a9523e-4e96-44f8-8117-600c60c24ed4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:51.966897 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a9523e-4e96-44f8-8117-600c60c24ed4 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:51.984208 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:51.986600 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-77a9523e-4e96-44f8-8117-600c60c24ed4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:51.986600 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:51.986600 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:51.986600 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:51.986600 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:51.986600 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:51.986600 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:51.986600 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:51.986600 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:51.986600 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:51.986600 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:51.986600 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:51.986600 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:51.986600 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:51.986600 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:51.987463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:51.987463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:51.987463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:51.987463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:51.987463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:51.987463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:51.987463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:51.987463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:51.987463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:51.987463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:51.987463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:51.987463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:51.987463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:51.987463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:51.987463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:51.987463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:51.987463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:51.988378 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:51.988378 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:51.988378 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:51.988378 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:51.988378 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 339/690] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:51.989276 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd825889-c065-4d57-807e-27304d8e7902 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:51.989351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd825889-c065-4d57-807e-27304d8e7902 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:51.989981 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd825889-c065-4d57-807e-27304d8e7902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:51.995708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9b49b-1c72-4d89-bfe8-95941fcd11da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:51.996394 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9b49b-1c72-4d89-bfe8-95941fcd11da None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:51.996671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9b49b-1c72-4d89-bfe8-95941fcd11da None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:51.996943 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9b49b-1c72-4d89-bfe8-95941fcd11da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:52.008353 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-cd825889-c065-4d57-807e-27304d8e7902 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:52.009837 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-cd825889-c065-4d57-807e-27304d8e7902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:52.079108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f9b49b-1c72-4d89-bfe8-95941fcd11da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:52.080661 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f9b49b-1c72-4d89-bfe8-95941fcd11da None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:52.103900 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 340/691] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:52.104790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd825889-c065-4d57-807e-27304d8e7902 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:52.193168 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 352/692] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:48:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:48:52.194031 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9b49b-1c72-4d89-bfe8-95941fcd11da None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:52.222427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd825889-c065-4d57-807e-27304d8e7902 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:52.228447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:52.248108 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:52.259880 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46a360-e742-46ba-84e6-83db44ce5b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:52.260756 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46a360-e742-46ba-84e6-83db44ce5b21 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:52.261137 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46a360-e742-46ba-84e6-83db44ce5b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:52.261532 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46a360-e742-46ba-84e6-83db44ce5b21 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:52.262348 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff46a360-e742-46ba-84e6-83db44ce5b21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:52.262751 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff46a360-e742-46ba-84e6-83db44ce5b21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:52.264369 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff46a360-e742-46ba-84e6-83db44ce5b21 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:52.277346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 341/693] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:52.278440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46a360-e742-46ba-84e6-83db44ce5b21 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:52.310158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9b49b-1c72-4d89-bfe8-95941fcd11da None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:52.350087 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0420ae-48f1-41f5-8515-ba050c95bc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:52.351227 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0420ae-48f1-41f5-8515-ba050c95bc12 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:52.351660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0420ae-48f1-41f5-8515-ba050c95bc12 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:52.352228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0420ae-48f1-41f5-8515-ba050c95bc12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:52.363695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0420ae-48f1-41f5-8515-ba050c95bc12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:52.365679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0420ae-48f1-41f5-8515-ba050c95bc12 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:52.383080 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46a360-e742-46ba-84e6-83db44ce5b21 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:52.385850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-dc0420ae-48f1-41f5-8515-ba050c95bc12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:52.385850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:52.385850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:52.385850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:52.385850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:52.385850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:52.385850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:52.385850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:52.385850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:52.385850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:52.385850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:52.385850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:52.385850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:52.385850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:52.385850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:52.388417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:52.388417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:52.388417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:52.388417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:52.388417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:52.388417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:52.388417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:52.388417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:52.388417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:52.388417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:52.388417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:52.388417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:52.388417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:52.388417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:52.388417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:52.388417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:52.388417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:52.389306 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:52.389306 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:52.389306 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:52.389306 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:52.390017 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 353/694] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:52.391084 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0420ae-48f1-41f5-8515-ba050c95bc12 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:52.419176 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b558a0-0535-429e-8941-e11e34dfd1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:52.420044 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b558a0-0535-429e-8941-e11e34dfd1c7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:52.420323 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b558a0-0535-429e-8941-e11e34dfd1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:52.420644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b558a0-0535-429e-8941-e11e34dfd1c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:52.432795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b558a0-0535-429e-8941-e11e34dfd1c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:52.433867 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b558a0-0535-429e-8941-e11e34dfd1c7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:52.451930 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-b7b558a0-0535-429e-8941-e11e34dfd1c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:52.451930 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:52.451930 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:52.451930 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:52.451930 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:52.451930 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:52.451930 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:52.451930 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:52.451930 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:52.451930 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:52.451930 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:52.451930 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:52.451930 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:52.451930 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:52.451930 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:52.453093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:52.453093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:52.453093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:52.453093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:52.453093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:52.453093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:52.453093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:52.453093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:52.453093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:52.453093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:52.453093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:52.453093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:52.453093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:52.453093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:52.453093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:52.453093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:52.453093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:52.454014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:52.454014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:52.454014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:52.454014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:52.454014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 342/695] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:52.454599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b558a0-0535-429e-8941-e11e34dfd1c7 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:52.491654 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:52.493486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:52.513576 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0420ae-48f1-41f5-8515-ba050c95bc12 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:52.554708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7696bbb-9aa7-4513-95c9-06bef2dbcadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:52.555505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7696bbb-9aa7-4513-95c9-06bef2dbcadd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:52.555781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7696bbb-9aa7-4513-95c9-06bef2dbcadd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:52.556058 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7696bbb-9aa7-4513-95c9-06bef2dbcadd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:52.563835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7696bbb-9aa7-4513-95c9-06bef2dbcadd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:52.565310 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7696bbb-9aa7-4513-95c9-06bef2dbcadd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:52.569293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b558a0-0535-429e-8941-e11e34dfd1c7 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:52.580499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-d7696bbb-9aa7-4513-95c9-06bef2dbcadd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 19:48:52.580499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:52.580499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:52.580499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:52.580499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:52.580499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:52.580499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:52.580499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:52.580499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:52.580499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:52.580499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:52.580499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:52.580499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:48:52.580499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:48:52.580499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:48:52.580499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:48:52.581478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:52.581478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:52.581478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:48:52.581478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:52.581478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:48:52.581478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:52.581478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:52.581478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:52.581478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:52.581478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:52.581478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:52.581478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:52.581478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:52.581478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:52.581478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:52.581478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:52.581478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:52.581478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:52.582446 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:52.582446 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:52.582446 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:48:52.582446 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:48:52.582446 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:48:52.582446 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:48:52.582446 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:48:52.582446 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:48:52.582446 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 19:48:52.582446 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:52.582446 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 354/696] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:48:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:52.584134 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7696bbb-9aa7-4513-95c9-06bef2dbcadd None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:52.612062 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b4584-f8b1-4c59-bb49-3e68813f08f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:52.613153 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b4584-f8b1-4c59-bb49-3e68813f08f3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:52.613717 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b4584-f8b1-4c59-bb49-3e68813f08f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:52.614234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b4584-f8b1-4c59-bb49-3e68813f08f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:52.615373 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b4584-f8b1-4c59-bb49-3e68813f08f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:52.615948 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b4584-f8b1-4c59-bb49-3e68813f08f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:52.617637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b4584-f8b1-4c59-bb49-3e68813f08f3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:52.632835 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 343/697] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:52.633955 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b4584-f8b1-4c59-bb49-3e68813f08f3 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:52.686536 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7696bbb-9aa7-4513-95c9-06bef2dbcadd None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:52.721468 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4c99a-ba6c-4475-bfac-b62d17265cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:52.722160 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4c99a-ba6c-4475-bfac-b62d17265cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:52.722439 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4c99a-ba6c-4475-bfac-b62d17265cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:52.722710 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4c99a-ba6c-4475-bfac-b62d17265cd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:52.723423 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a4c99a-ba6c-4475-bfac-b62d17265cd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:52.724270 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a4c99a-ba6c-4475-bfac-b62d17265cd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:52.725676 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a4c99a-ba6c-4475-bfac-b62d17265cd9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:52.737245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 355/698] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:52.738083 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4c99a-ba6c-4475-bfac-b62d17265cd9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:52.742288 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:52.747708 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:52.755199 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b4584-f8b1-4c59-bb49-3e68813f08f3 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:52.791998 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4dcfd9-5322-4fe5-b36d-d71b3fa45295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:52.793046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dcfd9-5322-4fe5-b36d-d71b3fa45295 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:52.793550 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dcfd9-5322-4fe5-b36d-d71b3fa45295 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:52.794229 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dcfd9-5322-4fe5-b36d-d71b3fa45295 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:52.795418 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4dcfd9-5322-4fe5-b36d-d71b3fa45295 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:52.795933 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4dcfd9-5322-4fe5-b36d-d71b3fa45295 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:52.797748 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4dcfd9-5322-4fe5-b36d-d71b3fa45295 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:52.818522 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 344/699] 10.222.0.44 () {56 vars in 1135 bytes} [Thu Jun 23 19:48:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:52.819707 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4dcfd9-5322-4fe5-b36d-d71b3fa45295 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:52.848052 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4c99a-ba6c-4475-bfac-b62d17265cd9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:52.893183 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf6bcb-cce3-49a6-961f-c377fd46446d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:52.894488 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf6bcb-cce3-49a6-961f-c377fd46446d None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:52.895185 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf6bcb-cce3-49a6-961f-c377fd46446d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:52.896603 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf6bcb-cce3-49a6-961f-c377fd46446d None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:52.897797 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bf6bcb-cce3-49a6-961f-c377fd46446d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:52.899865 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bf6bcb-cce3-49a6-961f-c377fd46446d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:52.936864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4dcfd9-5322-4fe5-b36d-d71b3fa45295 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:52.972262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70054f9a-d302-473d-a072-d933c6475b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:52.973073 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70054f9a-d302-473d-a072-d933c6475b37 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:52.973351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70054f9a-d302-473d-a072-d933c6475b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:52.973673 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70054f9a-d302-473d-a072-d933c6475b37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:52.986630 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 356/700] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jun 23 19:48:52 2022] POST /identity/v3/users => generated 254 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:48:52.987454 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf6bcb-cce3-49a6-961f-c377fd46446d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:52.987822 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70054f9a-d302-473d-a072-d933c6475b37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:52.989887 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70054f9a-d302-473d-a072-d933c6475b37 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:53.014940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-70054f9a-d302-473d-a072-d933c6475b37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:53.014940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:53.014940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:53.014940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:53.014940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:53.014940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:53.014940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:53.014940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:53.014940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:53.014940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:53.014940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:53.014940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:53.014940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:48:53.014940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:48:53.014940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:48:53.015946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:48:53.015946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:53.015946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:53.015946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:53.015946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:53.015946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:53.015946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:53.015946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:53.015946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:53.015946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:53.015946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:53.015946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:53.015946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:53.015946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:53.015946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:53.015946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:53.015946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:53.017426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:48:53.017426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:48:53.017426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:48:53.017426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:48:53.017426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:48:53.017426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:48:53.017426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:53.017426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:53.018210 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 345/701] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:48:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:53.028369 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70054f9a-d302-473d-a072-d933c6475b37 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:53.048651 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:53.158149 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70054f9a-d302-473d-a072-d933c6475b37 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:53.165743 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:53.205253 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a2952-bf73-4993-95b7-a52a8ed3d411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:53.206183 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a2952-bf73-4993-95b7-a52a8ed3d411 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:53.207160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a2952-bf73-4993-95b7-a52a8ed3d411 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:53.207662 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a2952-bf73-4993-95b7-a52a8ed3d411 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:53.208797 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a2952-bf73-4993-95b7-a52a8ed3d411 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:53.209309 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a2952-bf73-4993-95b7-a52a8ed3d411 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:53.210975 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a2952-bf73-4993-95b7-a52a8ed3d411 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:53.232590 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 346/702] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:48:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:53.235006 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a2952-bf73-4993-95b7-a52a8ed3d411 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:53.247756 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf6bcb-cce3-49a6-961f-c377fd46446d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:53.293410 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e912a-b606-443a-942b-f5825588d474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:53.293997 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e912a-b606-443a-942b-f5825588d474 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:53.294281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e912a-b606-443a-942b-f5825588d474 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:53.294557 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e912a-b606-443a-942b-f5825588d474 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:53.319885 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:53.352700 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a2952-bf73-4993-95b7-a52a8ed3d411 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:53.391130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e912a-b606-443a-942b-f5825588d474 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:53.392449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c89e3-21b8-45f2-911e-c98fc441d7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:53.392836 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e912a-b606-443a-942b-f5825588d474 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:53.393701 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c89e3-21b8-45f2-911e-c98fc441d7ec None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:53.394180 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c89e3-21b8-45f2-911e-c98fc441d7ec None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:53.394713 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c89e3-21b8-45f2-911e-c98fc441d7ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:53.395902 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c89e3-21b8-45f2-911e-c98fc441d7ec None admin] RBAC: Adding query filter params (role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb, scope.project.id=155180ba63c446ca889a6eb3f94cc31a, user.id=a63d1d902c024c97a38f7d92f73814d3) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:53.396488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c89e3-21b8-45f2-911e-c98fc441d7ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:53.398268 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319c89e3-21b8-45f2-911e-c98fc441d7ec None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:53.406297 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 347/703] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:48:53 2022] GET /identity/v3/role_assignments?user.id=a63d1d902c024c97a38f7d92f73814d3&scope.project.id=155180ba63c446ca889a6eb3f94cc31a&role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:53.459721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:53.480141 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e2110-a39e-41fa-826d-50a483d7ceca None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:53.480738 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e2110-a39e-41fa-826d-50a483d7ceca None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:53.481178 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e2110-a39e-41fa-826d-50a483d7ceca None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:53.484516 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 348/704] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:53.494837 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d333cb-e04f-4f42-87ad-a7cfce11ee04 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:53.495667 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d333cb-e04f-4f42-87ad-a7cfce11ee04 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:53.496094 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d333cb-e04f-4f42-87ad-a7cfce11ee04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:53.510354 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 357/705] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:48:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:48:53.515761 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e912a-b606-443a-942b-f5825588d474 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:53.521899 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c8d333cb-e04f-4f42-87ad-a7cfce11ee04 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:53.523292 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c8d333cb-e04f-4f42-87ad-a7cfce11ee04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:53.610935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 349/706] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 19:48:53.633440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d333cb-e04f-4f42-87ad-a7cfce11ee04 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:53.637496 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e912a-b606-443a-942b-f5825588d474 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:53.675672 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f344e-0636-4f9e-b249-7433e4af1560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:53.677116 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f344e-0636-4f9e-b249-7433e4af1560 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:53.677463 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f344e-0636-4f9e-b249-7433e4af1560 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:53.677803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f344e-0636-4f9e-b249-7433e4af1560 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:53.689525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004f344e-0636-4f9e-b249-7433e4af1560 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:53.691180 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004f344e-0636-4f9e-b249-7433e4af1560 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:53.707746 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:53.712282 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-004f344e-0636-4f9e-b249-7433e4af1560 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:53.712282 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:53.712282 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:53.712282 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:53.712282 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:53.712282 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:53.712282 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:53.712282 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:53.712282 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:53.712282 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:53.712282 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:53.712282 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:53.712282 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:53.712282 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:53.712282 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:53.717494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:53.717494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:53.717494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:53.717494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:53.717494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:53.717494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:53.717494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:53.717494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:53.717494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:53.717494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:53.717494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:53.717494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:53.717494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:53.717494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:53.717494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:53.717494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:53.717494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:53.718428 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:53.718428 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:53.718428 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:53.718428 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:53.718428 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 358/707] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:48:53.719320 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f344e-0636-4f9e-b249-7433e4af1560 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:53.736552 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:53.756853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d333cb-e04f-4f42-87ad-a7cfce11ee04 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:53.818202 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd32fd9-ef55-4b75-8db5-d801da926499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:53.819388 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd32fd9-ef55-4b75-8db5-d801da926499 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:53.819903 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd32fd9-ef55-4b75-8db5-d801da926499 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:53.820401 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd32fd9-ef55-4b75-8db5-d801da926499 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:53.821269 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd32fd9-ef55-4b75-8db5-d801da926499 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:53.822870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd32fd9-ef55-4b75-8db5-d801da926499 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:53.830464 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f344e-0636-4f9e-b249-7433e4af1560 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:53.854191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-8fd32fd9-ef55-4b75-8db5-d801da926499 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 19:48:53.854191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:53.854191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 19:48:53.854191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 19:48:53.854191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 19:48:53.854191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 19:48:53.854191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 19:48:53.854191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:48:53.854191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 19:48:53.854191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:53.854191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:48:53.854191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:53.854191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:53.854191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:53.854191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:53.854191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:53.855234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:53.855234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:53.855234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:53.855234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:53.855234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:53.855234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:53.855234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:53.855234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 19:48:53.855234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 19:48:53.855234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 19:48:53.855234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 19:48:53.855234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:53.855234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:53.855234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:53.855234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:53.855234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:53.856092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:53.856092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:53.856092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:53.856092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:53.856092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:53.856092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:53.856092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:53.856092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:53.856092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:53.856092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:53.856092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:53.856092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 19:48:53.856092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:48:53.856092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 19:48:53.856092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:53.859081 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 350/708] 10.222.0.44 () {56 vars in 1129 bytes} [Thu Jun 23 19:48:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:48:53.866006 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd32fd9-ef55-4b75-8db5-d801da926499 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:53.867522 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32014d92-269d-4057-be86-ee12e8076525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:53.868256 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32014d92-269d-4057-be86-ee12e8076525 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:53.868662 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32014d92-269d-4057-be86-ee12e8076525 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:53.869004 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32014d92-269d-4057-be86-ee12e8076525 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:53.869792 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32014d92-269d-4057-be86-ee12e8076525 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:53.870178 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32014d92-269d-4057-be86-ee12e8076525 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:53.872277 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32014d92-269d-4057-be86-ee12e8076525 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:53.888395 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 359/709] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:53.890203 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32014d92-269d-4057-be86-ee12e8076525 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:53.966121 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:53.998641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32014d92-269d-4057-be86-ee12e8076525 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:54.040116 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de03c88-3e3a-4aac-a2d6-a7c7dc11ff1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:54.042122 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de03c88-3e3a-4aac-a2d6-a7c7dc11ff1b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:54.042810 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de03c88-3e3a-4aac-a2d6-a7c7dc11ff1b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:54.044186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de03c88-3e3a-4aac-a2d6-a7c7dc11ff1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:54.060349 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de03c88-3e3a-4aac-a2d6-a7c7dc11ff1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:54.062175 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de03c88-3e3a-4aac-a2d6-a7c7dc11ff1b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:54.084018 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-2de03c88-3e3a-4aac-a2d6-a7c7dc11ff1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:54.084018 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:54.084018 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:54.084018 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:54.084018 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:54.084018 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:54.084018 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:54.084018 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:54.084018 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:54.084018 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:54.084018 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:54.084018 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:54.084018 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:48:54.084018 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:48:54.084018 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:48:54.085194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:48:54.085194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:54.085194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:54.085194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:54.085194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:54.085194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:54.085194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:54.085194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:54.085194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:54.085194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:54.085194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:54.085194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:54.085194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:54.085194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:54.085194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:54.085194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:54.085194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:54.086293 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:48:54.086293 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:48:54.086293 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:48:54.086293 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:48:54.086293 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:48:54.086293 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:48:54.086293 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:54.086293 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:54.088513 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 360/710] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:48:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:54.096224 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de03c88-3e3a-4aac-a2d6-a7c7dc11ff1b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:54.123010 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd32fd9-ef55-4b75-8db5-d801da926499 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:54.160288 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:54.165816 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b34993-6a57-40b5-ade7-386c03e7f771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:54.166486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b34993-6a57-40b5-ade7-386c03e7f771 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:54.166770 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b34993-6a57-40b5-ade7-386c03e7f771 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:54.167042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b34993-6a57-40b5-ade7-386c03e7f771 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:54.167750 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b34993-6a57-40b5-ade7-386c03e7f771 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:54.168054 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b34993-6a57-40b5-ade7-386c03e7f771 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:54.169479 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b34993-6a57-40b5-ade7-386c03e7f771 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:54.176037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 351/711] 10.222.0.44 () {56 vars in 1124 bytes} [Thu Jun 23 19:48:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:54.176037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f5bc3-3dd7-4ca7-b6c3-317e226e9e78 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:54.176037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f5bc3-3dd7-4ca7-b6c3-317e226e9e78 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:54.180496 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f5bc3-3dd7-4ca7-b6c3-317e226e9e78 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:54.182123 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 352/712] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:54.183013 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f5bc3-3dd7-4ca7-b6c3-317e226e9e78 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:54.231132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de03c88-3e3a-4aac-a2d6-a7c7dc11ff1b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:54.266584 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:54.273056 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc419a-a2cf-4d55-af38-ab6ac224643c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:54.273972 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc419a-a2cf-4d55-af38-ab6ac224643c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:54.274452 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc419a-a2cf-4d55-af38-ab6ac224643c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:54.274969 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc419a-a2cf-4d55-af38-ab6ac224643c None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:54.276183 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcc419a-a2cf-4d55-af38-ab6ac224643c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:54.276783 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcc419a-a2cf-4d55-af38-ab6ac224643c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:54.278553 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcc419a-a2cf-4d55-af38-ab6ac224643c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:54.292360 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 361/713] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:48:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:54.292851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f5bc3-3dd7-4ca7-b6c3-317e226e9e78 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:54.296512 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b617755-7988-4fef-b864-168ab6979c66 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:54.296512 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b617755-7988-4fef-b864-168ab6979c66 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:54.296512 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b617755-7988-4fef-b864-168ab6979c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:54.313256 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-2b617755-7988-4fef-b864-168ab6979c66 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:54.314853 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2b617755-7988-4fef-b864-168ab6979c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:54.328746 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdeef4-d443-435f-bda5-32e21cd4385a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:54.329413 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdeef4-d443-435f-bda5-32e21cd4385a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:54.329696 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdeef4-d443-435f-bda5-32e21cd4385a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:54.329972 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdeef4-d443-435f-bda5-32e21cd4385a None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:54.330672 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fdeef4-d443-435f-bda5-32e21cd4385a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:54.330975 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fdeef4-d443-435f-bda5-32e21cd4385a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:54.332395 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fdeef4-d443-435f-bda5-32e21cd4385a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:54.343519 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 353/714] 10.222.0.44 () {56 vars in 1124 bytes} [Thu Jun 23 19:48:54 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:54.344550 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdeef4-d443-435f-bda5-32e21cd4385a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:54.438171 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 362/715] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:54.478741 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:54.483886 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b617755-7988-4fef-b864-168ab6979c66 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:54.527283 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdeef4-d443-435f-bda5-32e21cd4385a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:54.594786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9505e-9707-479e-8c5e-1ff77b8f7920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:54.595447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9505e-9707-479e-8c5e-1ff77b8f7920 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:54.595732 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9505e-9707-479e-8c5e-1ff77b8f7920 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:54.596004 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9505e-9707-479e-8c5e-1ff77b8f7920 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:54.596870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b9505e-9707-479e-8c5e-1ff77b8f7920 None admin] RBAC: Adding query filter params (role.id=5f27c72acb19431fa3223c962bc42938, scope.project.id=155180ba63c446ca889a6eb3f94cc31a, user.id=96db03079816461bb322633f64b9fb9d) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:54.597180 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b9505e-9707-479e-8c5e-1ff77b8f7920 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:54.599399 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b9505e-9707-479e-8c5e-1ff77b8f7920 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:54.612249 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 354/716] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:48:54 2022] GET /identity/v3/role_assignments?user.id=96db03079816461bb322633f64b9fb9d&scope.project.id=155180ba63c446ca889a6eb3f94cc31a&role.id=5f27c72acb19431fa3223c962bc42938 => generated 608 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:54.667158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b617755-7988-4fef-b864-168ab6979c66 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:54.711742 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:54.726351 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39384769-b17c-42ce-9dc3-fed57d0af3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:54.727153 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39384769-b17c-42ce-9dc3-fed57d0af3d4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:54.727432 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39384769-b17c-42ce-9dc3-fed57d0af3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:54.727704 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39384769-b17c-42ce-9dc3-fed57d0af3d4 None admin] PATH_INFO: `/v3/services/network` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:54.728382 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39384769-b17c-42ce-9dc3-fed57d0af3d4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:54.730545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39384769-b17c-42ce-9dc3-fed57d0af3d4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:54.751081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-39384769-b17c-42ce-9dc3-fed57d0af3d4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 19:48:54.751081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:54.751081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 19:48:54.751081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 19:48:54.751081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 19:48:54.751081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 19:48:54.751081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 19:48:54.751081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:48:54.751081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 19:48:54.751081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:54.751081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:48:54.751081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:54.751081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:54.751081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:54.751081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:54.751081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:54.751081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:54.752151 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:54.752151 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:54.752151 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:54.752151 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:54.752151 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:54.752151 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:54.752151 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 19:48:54.752151 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 19:48:54.752151 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 19:48:54.752151 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 19:48:54.752151 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:54.752151 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:54.752151 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:54.752151 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:54.752151 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:54.752151 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:54.753035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:54.753035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:54.753035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:54.753035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:54.753035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:54.753035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:54.753035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:54.753035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:54.753035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:54.753035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:54.753035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 19:48:54.753035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:48:54.753035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 19:48:54.753035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:54.753035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 363/717] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:48:54 2022] GET /identity/v3/services/network => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:54.765486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9505e-9707-479e-8c5e-1ff77b8f7920 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:54.899961 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9505e-9707-479e-8c5e-1ff77b8f7920 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:54.923730 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:54.935480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae9905-4d0b-49cc-b79f-0ac1d05b0b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:54.941538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae9905-4d0b-49cc-b79f-0ac1d05b0b00 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:54.941538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae9905-4d0b-49cc-b79f-0ac1d05b0b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:54.941538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae9905-4d0b-49cc-b79f-0ac1d05b0b00 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:54.941538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae9905-4d0b-49cc-b79f-0ac1d05b0b00 None admin] RBAC: Adding query filter params (name=network) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:54.941538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae9905-4d0b-49cc-b79f-0ac1d05b0b00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:54.941538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae9905-4d0b-49cc-b79f-0ac1d05b0b00 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:54.945854 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 355/718] 10.222.0.44 () {56 vars in 1112 bytes} [Thu Jun 23 19:48:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:54.951101 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39384769-b17c-42ce-9dc3-fed57d0af3d4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:55.060516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39384769-b17c-42ce-9dc3-fed57d0af3d4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:55.096023 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d7d06-3749-47d2-b641-91710d482efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:55.096739 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d7d06-3749-47d2-b641-91710d482efa None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:55.097019 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d7d06-3749-47d2-b641-91710d482efa None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:55.097291 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d7d06-3749-47d2-b641-91710d482efa None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:55.098013 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d7d06-3749-47d2-b641-91710d482efa None admin] RBAC: Adding query filter params (type=network) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:55.098317 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d7d06-3749-47d2-b641-91710d482efa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:55.099889 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d7d06-3749-47d2-b641-91710d482efa None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:55.111608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 364/719] 10.222.0.44 () {56 vars in 1112 bytes} [Thu Jun 23 19:48:54 2022] GET /identity/v3/services?type=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:55.588908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee7501-d4f0-44d6-b3fe-ebff3422d9fc None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:55.589434 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee7501-d4f0-44d6-b3fe-ebff3422d9fc None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:55.589434 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dee7501-d4f0-44d6-b3fe-ebff3422d9fc None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:55.591386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 356/720] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:55.596757 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8593c5a-8dcc-4ce7-aa80-4f64c8cac3be None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:55.597189 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8593c5a-8dcc-4ce7-aa80-4f64c8cac3be None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:55.597494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8593c5a-8dcc-4ce7-aa80-4f64c8cac3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:55.629306 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e8593c5a-8dcc-4ce7-aa80-4f64c8cac3be None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:55.630553 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e8593c5a-8dcc-4ce7-aa80-4f64c8cac3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:55.700040 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:55.727569 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 365/721] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:55.764187 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee7501-d4f0-44d6-b3fe-ebff3422d9fc None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:55.949349 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dee7501-d4f0-44d6-b3fe-ebff3422d9fc None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:55.991719 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e60e39-7dbd-46f9-abc8-2f8a4ae0e0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:55.992914 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e60e39-7dbd-46f9-abc8-2f8a4ae0e0b7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:55.993408 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e60e39-7dbd-46f9-abc8-2f8a4ae0e0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:55.993888 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e60e39-7dbd-46f9-abc8-2f8a4ae0e0b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:56.008997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e60e39-7dbd-46f9-abc8-2f8a4ae0e0b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:56.016957 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-65e60e39-7dbd-46f9-abc8-2f8a4ae0e0b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:56.016957 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:56.016957 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:56.016957 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:56.016957 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:56.016957 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:56.016957 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:56.016957 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:56.016957 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:56.016957 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:56.016957 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:56.016957 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:56.016957 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:56.016957 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:56.016957 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:56.016957 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:56.017877 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:56.017877 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:56.017877 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:56.017877 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:56.017877 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:56.017877 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:56.017877 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:56.017877 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:56.017877 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:56.017877 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:56.017877 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:56.017877 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:56.017877 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:56.017877 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:56.017877 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:56.017877 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:56.018722 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:56.018722 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:56.018722 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:56.018722 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:56.018722 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:56.018722 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:56.018722 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:56.018722 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:56.018722 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:56.018722 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:56.018722 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:48:56.018722 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:56.025223 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 357/722] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jun 23 19:48:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:56.031059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8593c5a-8dcc-4ce7-aa80-4f64c8cac3be None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:56.159900 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8593c5a-8dcc-4ce7-aa80-4f64c8cac3be None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:56.195266 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f4690-f37b-48bd-a7d0-c2a51ed58302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:56.195939 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f4690-f37b-48bd-a7d0-c2a51ed58302 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:56.196221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f4690-f37b-48bd-a7d0-c2a51ed58302 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:56.196522 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f4690-f37b-48bd-a7d0-c2a51ed58302 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:56.197271 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f4690-f37b-48bd-a7d0-c2a51ed58302 None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:56.197573 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f4690-f37b-48bd-a7d0-c2a51ed58302 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:56.198952 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f4690-f37b-48bd-a7d0-c2a51ed58302 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:56.208468 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 366/723] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jun 23 19:48:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:56.213505 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e60e39-7dbd-46f9-abc8-2f8a4ae0e0b7 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:56.262601 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7892c-0000-451c-bc51-29323786ddf6 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:56.262985 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7892c-0000-451c-bc51-29323786ddf6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:56.263341 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7892c-0000-451c-bc51-29323786ddf6 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:56.265229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 367/724] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:48:56.270258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8130f-e7b7-48b4-84e2-bd918acc8611 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:56.270636 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8130f-e7b7-48b4-84e2-bd918acc8611 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:56.270990 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8130f-e7b7-48b4-84e2-bd918acc8611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:56.304092 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-12c8130f-e7b7-48b4-84e2-bd918acc8611 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:56.305313 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-12c8130f-e7b7-48b4-84e2-bd918acc8611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:56.332536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:56.388662 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:56.423671 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e60e39-7dbd-46f9-abc8-2f8a4ae0e0b7 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:56.451313 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 368/725] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 19:48:56.483821 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8e7aa-f766-4230-a81f-51f0e0193eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:56.484580 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8e7aa-f766-4230-a81f-51f0e0193eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:56.484867 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8e7aa-f766-4230-a81f-51f0e0193eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:56.485143 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8e7aa-f766-4230-a81f-51f0e0193eb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:56.498758 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8e7aa-f766-4230-a81f-51f0e0193eb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:56.500222 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8e7aa-f766-4230-a81f-51f0e0193eb8 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:56.524895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c8130f-e7b7-48b4-84e2-bd918acc8611 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:56.525584 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-56c8e7aa-f766-4230-a81f-51f0e0193eb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:56.525584 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:56.525584 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:56.525584 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:56.525584 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:56.525584 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:56.525584 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:56.525584 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:56.525584 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:56.525584 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:56.525584 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:56.525584 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:56.525584 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:56.525584 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:56.525584 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:56.526465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:56.526465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:56.526465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:56.526465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:56.526465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:56.526465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:56.526465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:56.526465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:56.526465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:56.526465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:56.526465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:56.526465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:56.526465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:56.526465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:56.526465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:56.526465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:56.526465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:56.527371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:56.527371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:56.527371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:56.527371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:56.527371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 358/726] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:56.534890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8e7aa-f766-4230-a81f-51f0e0193eb8 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:56.686084 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8e7aa-f766-4230-a81f-51f0e0193eb8 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:56.704341 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:56.706485 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c8130f-e7b7-48b4-84e2-bd918acc8611 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:56.744211 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b467be6-9f5d-4198-9b87-1628fff45746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:56.744958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b467be6-9f5d-4198-9b87-1628fff45746 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:56.745244 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b467be6-9f5d-4198-9b87-1628fff45746 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:56.745516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b467be6-9f5d-4198-9b87-1628fff45746 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:56.756099 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b467be6-9f5d-4198-9b87-1628fff45746 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:56.763693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-8b467be6-9f5d-4198-9b87-1628fff45746 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:48:56.763693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:56.763693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:56.763693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:56.763693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:56.763693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:56.763693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:56.763693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:56.763693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:56.763693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:56.763693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:56.763693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:56.763693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:56.763693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:56.763693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:56.763693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:56.764831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:56.764831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:56.764831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:56.764831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:56.764831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:56.764831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:56.764831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:56.764831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:56.764831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:56.764831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:56.764831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:56.764831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:56.764831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:56.764831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:56.764831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:56.764831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:56.765875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:56.765875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:56.765875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:56.765875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:56.765875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:56.765875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:56.765875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:56.765875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:56.765875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:56.765875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:56.765875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:48:56.765875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:56.766574 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc75332-6216-4b47-8fa1-49d12d295535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:56.766671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 369/727] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:48:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:48:56.766741 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc75332-6216-4b47-8fa1-49d12d295535 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:56.766741 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc75332-6216-4b47-8fa1-49d12d295535 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:56.766741 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc75332-6216-4b47-8fa1-49d12d295535 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:56.767096 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc75332-6216-4b47-8fa1-49d12d295535 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:56.767409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc75332-6216-4b47-8fa1-49d12d295535 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:56.768986 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc75332-6216-4b47-8fa1-49d12d295535 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:56.771546 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b467be6-9f5d-4198-9b87-1628fff45746 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:56.796348 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 359/728] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:56.801717 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc75332-6216-4b47-8fa1-49d12d295535 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:56.821685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:56.891979 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b467be6-9f5d-4198-9b87-1628fff45746 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:56.928042 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f6177-7014-459f-b5bd-cf9b881e258d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:56.928720 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f6177-7014-459f-b5bd-cf9b881e258d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:56.929006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f6177-7014-459f-b5bd-cf9b881e258d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:56.929282 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f6177-7014-459f-b5bd-cf9b881e258d None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:56.930018 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f6177-7014-459f-b5bd-cf9b881e258d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:56.930327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f6177-7014-459f-b5bd-cf9b881e258d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:56.931702 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f6177-7014-459f-b5bd-cf9b881e258d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:56.932258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc75332-6216-4b47-8fa1-49d12d295535 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:56.945355 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 370/729] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:48:56.946627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71e2bf-ada0-49d0-a205-455fe6728377 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:56.946921 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71e2bf-ada0-49d0-a205-455fe6728377 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:56.947207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71e2bf-ada0-49d0-a205-455fe6728377 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:56.948959 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 371/730] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:56.949448 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d71e2bf-ada0-49d0-a205-455fe6728377 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:56.970973 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7358707a-a7e4-4520-b502-a80fd90e509c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:56.972177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358707a-a7e4-4520-b502-a80fd90e509c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:56.972682 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358707a-a7e4-4520-b502-a80fd90e509c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:56.973152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358707a-a7e4-4520-b502-a80fd90e509c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:56.981003 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7358707a-a7e4-4520-b502-a80fd90e509c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:56.982722 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7358707a-a7e4-4520-b502-a80fd90e509c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:56.998896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-7358707a-a7e4-4520-b502-a80fd90e509c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 19:48:56.998896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:56.998896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:56.998896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:56.998896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:56.998896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:56.998896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:56.998896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:56.998896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:56.998896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:56.998896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:56.998896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:56.998896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:48:56.998896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:48:56.998896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:48:56.998896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:48:56.999803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:56.999803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:56.999803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:48:56.999803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:56.999803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:48:56.999803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:56.999803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:56.999803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:56.999803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:56.999803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:56.999803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:56.999803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:56.999803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:56.999803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:56.999803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:56.999803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:56.999803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:56.999803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:57.000776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:57.000776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:57.000776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:48:57.000776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:48:57.000776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:48:57.000776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:48:57.000776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:48:57.000776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:48:57.000776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 19:48:57.000776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:57.003308 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 360/731] 10.222.0.44 () {56 vars in 1117 bytes} [Thu Jun 23 19:48:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:57.009684 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ebe2f-8b7d-4d4b-b875-71a7ea229f8b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:57.010195 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ebe2f-8b7d-4d4b-b875-71a7ea229f8b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:57.010717 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ebe2f-8b7d-4d4b-b875-71a7ea229f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:57.028093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3b4ebe2f-8b7d-4d4b-b875-71a7ea229f8b None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:57.029750 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3b4ebe2f-8b7d-4d4b-b875-71a7ea229f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:57.044814 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:57.046177 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:57.054276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d71e2bf-ada0-49d0-a205-455fe6728377 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:57.093077 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5336e-adeb-472a-9df5-46cba2f2a3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:57.093880 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5336e-adeb-472a-9df5-46cba2f2a3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:57.094164 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5336e-adeb-472a-9df5-46cba2f2a3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:57.094443 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5336e-adeb-472a-9df5-46cba2f2a3c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:57.105607 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5336e-adeb-472a-9df5-46cba2f2a3c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:57.107069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5336e-adeb-472a-9df5-46cba2f2a3c9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:57.108777 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 361/732] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:57.110521 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ebe2f-8b7d-4d4b-b875-71a7ea229f8b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:57.126744 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-f0e5336e-adeb-472a-9df5-46cba2f2a3c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:57.126744 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:57.126744 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:57.126744 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:57.126744 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:57.126744 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:57.126744 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:57.126744 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:57.126744 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:57.126744 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:57.126744 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:57.126744 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:57.126744 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:57.126744 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:57.126744 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:57.127708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:57.127708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:57.127708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:57.127708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:57.127708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:57.127708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:57.127708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:57.127708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:57.127708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:57.127708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:57.127708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:57.127708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:57.127708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:57.127708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:57.127708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:57.127708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:57.127708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:57.129235 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:57.129235 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:57.129235 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:57.129235 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:57.129235 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 372/733] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:57.133660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5336e-adeb-472a-9df5-46cba2f2a3c9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:57.223906 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ebe2f-8b7d-4d4b-b875-71a7ea229f8b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:57.253822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5336e-adeb-472a-9df5-46cba2f2a3c9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:57.261762 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e43c0-2455-458e-9f68-cd4ad6d95606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:57.262756 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e43c0-2455-458e-9f68-cd4ad6d95606 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:57.263235 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e43c0-2455-458e-9f68-cd4ad6d95606 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:57.263782 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e43c0-2455-458e-9f68-cd4ad6d95606 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:57.264989 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698e43c0-2455-458e-9f68-cd4ad6d95606 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:57.265526 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698e43c0-2455-458e-9f68-cd4ad6d95606 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:57.267158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698e43c0-2455-458e-9f68-cd4ad6d95606 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:57.291655 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 362/734] 10.222.0.44 () {56 vars in 1149 bytes} [Thu Jun 23 19:48:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:57.291655 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e43c0-2455-458e-9f68-cd4ad6d95606 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:57.297924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2336b069-e987-4739-a14c-b360d94ad61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:57.298598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336b069-e987-4739-a14c-b360d94ad61d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:57.298882 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336b069-e987-4739-a14c-b360d94ad61d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:57.299161 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336b069-e987-4739-a14c-b360d94ad61d None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:57.299903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2336b069-e987-4739-a14c-b360d94ad61d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:57.300212 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2336b069-e987-4739-a14c-b360d94ad61d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:57.302224 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2336b069-e987-4739-a14c-b360d94ad61d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:57.302981 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:57.320279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 373/735] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:57.321503 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2336b069-e987-4739-a14c-b360d94ad61d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:57.331302 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:57.401850 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698e43c0-2455-458e-9f68-cd4ad6d95606 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:57.439837 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64426a8-c8fb-4119-90c7-b5675ded61e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:57.440831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64426a8-c8fb-4119-90c7-b5675ded61e7 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:57.441290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64426a8-c8fb-4119-90c7-b5675ded61e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:57.441773 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64426a8-c8fb-4119-90c7-b5675ded61e7 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:57.442744 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64426a8-c8fb-4119-90c7-b5675ded61e7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:57.444451 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64426a8-c8fb-4119-90c7-b5675ded61e7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:57.452248 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2336b069-e987-4739-a14c-b360d94ad61d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:57.573953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438ff3c-6091-4500-8a15-e9dae963f55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:57.574610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438ff3c-6091-4500-8a15-e9dae963f55a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:57.574891 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438ff3c-6091-4500-8a15-e9dae963f55a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:57.575166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438ff3c-6091-4500-8a15-e9dae963f55a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:57.593348 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 363/736] 10.222.0.44 () {60 vars in 1141 bytes} [Thu Jun 23 19:48:57 2022] POST /identity/v3/services => generated 251 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:48:57.594016 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64426a8-c8fb-4119-90c7-b5675ded61e7 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:57.688903 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:57.691058 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438ff3c-6091-4500-8a15-e9dae963f55a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:57.693234 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438ff3c-6091-4500-8a15-e9dae963f55a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:57.702106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:57.734377 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64426a8-c8fb-4119-90c7-b5675ded61e7 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:57.775984 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecb0e17-3575-4d6f-94fa-3a9901869496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:57.776870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb0e17-3575-4d6f-94fa-3a9901869496 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:57.777153 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb0e17-3575-4d6f-94fa-3a9901869496 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:57.777429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecb0e17-3575-4d6f-94fa-3a9901869496 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:57.789287 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb0e17-3575-4d6f-94fa-3a9901869496 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:57.791790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecb0e17-3575-4d6f-94fa-3a9901869496 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:57.808870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-fecb0e17-3575-4d6f-94fa-3a9901869496 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 19:48:57.808870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:57.808870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:57.808870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:57.808870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:57.808870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:57.808870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:57.808870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:57.808870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:57.808870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:57.808870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:57.808870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:57.808870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:48:57.808870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:48:57.808870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:48:57.808870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:48:57.809878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:57.809878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:57.809878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:48:57.809878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:57.809878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:48:57.809878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:57.809878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:57.809878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:57.809878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:57.809878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:57.809878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:57.809878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:57.809878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:57.809878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:57.809878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:57.809878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:57.809878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:57.809878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:57.811161 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:57.811161 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:57.811161 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:48:57.811161 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:48:57.811161 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:48:57.811161 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:48:57.811161 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:48:57.811161 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:48:57.811161 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 19:48:57.811161 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:57.811161 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 364/737] 10.222.0.44 () {56 vars in 1093 bytes} [Thu Jun 23 19:48:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:57.811880 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b117d9-30f5-430f-85ed-e3107c3595a0 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:57.811880 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b117d9-30f5-430f-85ed-e3107c3595a0 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:57.811880 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b117d9-30f5-430f-85ed-e3107c3595a0 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:57.813395 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 365/738] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:57.814444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f32997-c968-4b2a-9a58-57db1534dc1f None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:57.814731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f32997-c968-4b2a-9a58-57db1534dc1f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:57.815007 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f32997-c968-4b2a-9a58-57db1534dc1f None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:57.819728 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 366/739] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:57.819728 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f32997-c968-4b2a-9a58-57db1534dc1f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:57.824684 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 374/740] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:48:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 19:48:57.826698 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa03b3d-f630-4a44-90a7-87bc54a81b8d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:57.827128 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa03b3d-f630-4a44-90a7-87bc54a81b8d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:57.827544 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa03b3d-f630-4a44-90a7-87bc54a81b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:57.844598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5fa03b3d-f630-4a44-90a7-87bc54a81b8d None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:57.845802 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5fa03b3d-f630-4a44-90a7-87bc54a81b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:57.929532 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f32997-c968-4b2a-9a58-57db1534dc1f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:57.954631 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:57.967145 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf78b69-2b92-47a1-b046-c3a6936b6977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:57.970159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf78b69-2b92-47a1-b046-c3a6936b6977 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:57.970159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf78b69-2b92-47a1-b046-c3a6936b6977 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:57.970159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf78b69-2b92-47a1-b046-c3a6936b6977 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:57.970159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf78b69-2b92-47a1-b046-c3a6936b6977 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:57.970159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf78b69-2b92-47a1-b046-c3a6936b6977 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:57.971152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf78b69-2b92-47a1-b046-c3a6936b6977 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:57.996200 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 367/741] 10.222.0.44 () {56 vars in 1133 bytes} [Thu Jun 23 19:48:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:57.997507 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed83a1-50ce-4712-824a-7802a6759145 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:57.997793 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed83a1-50ce-4712-824a-7802a6759145 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:57.998062 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed83a1-50ce-4712-824a-7802a6759145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:58.015044 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b6ed83a1-50ce-4712-824a-7802a6759145 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:58.016233 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b6ed83a1-50ce-4712-824a-7802a6759145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:58.028474 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 375/742] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:57 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:58.029317 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa03b3d-f630-4a44-90a7-87bc54a81b8d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:58.075037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:58.117224 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 368/743] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:57 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 19:48:58.118799 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed83a1-50ce-4712-824a-7802a6759145 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:58.169664 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa03b3d-f630-4a44-90a7-87bc54a81b8d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:58.205746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030fb88-804a-4cdb-8328-5e8114e4c83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:58.206598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030fb88-804a-4cdb-8328-5e8114e4c83f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:58.206868 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030fb88-804a-4cdb-8328-5e8114e4c83f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:58.207132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030fb88-804a-4cdb-8328-5e8114e4c83f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:58.217770 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4030fb88-804a-4cdb-8328-5e8114e4c83f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:58.219384 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4030fb88-804a-4cdb-8328-5e8114e4c83f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:58.240658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed83a1-50ce-4712-824a-7802a6759145 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:58.242220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-4030fb88-804a-4cdb-8328-5e8114e4c83f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:58.242220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:58.242220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:58.242220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:58.242220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:58.242220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:58.242220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:58.242220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:58.242220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:58.242220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:58.242220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:58.242220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:58.242220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:58.242220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:58.242220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:58.243071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:58.243071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:58.243071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:58.243071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:58.243071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:58.243071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:58.243071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:58.243071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:58.243071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:58.243071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:58.243071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:58.243071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:58.243071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:58.243071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:58.243071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:58.243071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:58.243071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:58.243990 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:58.243990 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:58.243990 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:58.243990 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:58.243990 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 376/744] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:58.245559 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030fb88-804a-4cdb-8328-5e8114e4c83f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:58.256908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:58.280032 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9350268-0835-4637-8385-166f131ca802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:58.280849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9350268-0835-4637-8385-166f131ca802 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:58.281821 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9350268-0835-4637-8385-166f131ca802 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:58.281821 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9350268-0835-4637-8385-166f131ca802 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:58.291357 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9350268-0835-4637-8385-166f131ca802 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:58.292861 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9350268-0835-4637-8385-166f131ca802 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:58.312004 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-a9350268-0835-4637-8385-166f131ca802 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:58.312004 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:58.312004 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:58.312004 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:58.312004 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:58.312004 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:58.312004 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:58.312004 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:58.312004 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:58.312004 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:58.312004 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:58.312004 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:58.312004 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:58.312004 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:58.312004 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:58.313272 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:58.313272 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:58.313272 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:58.313272 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:58.313272 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:58.313272 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:58.313272 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:58.313272 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:58.313272 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:58.313272 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:58.313272 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:58.313272 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:58.313272 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:58.313272 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:58.313272 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:58.313272 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:58.313272 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:58.314159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:58.314159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:58.314159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:58.314159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:58.314159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 369/745] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:58.314443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9350268-0835-4637-8385-166f131ca802 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:58.341760 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:58.354118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030fb88-804a-4cdb-8328-5e8114e4c83f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:58.392507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b538f-7a44-4b93-9ed0-b7724291dca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:58.393176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b538f-7a44-4b93-9ed0-b7724291dca2 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:58.393857 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b538f-7a44-4b93-9ed0-b7724291dca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:58.394145 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b538f-7a44-4b93-9ed0-b7724291dca2 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:58.394920 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7b538f-7a44-4b93-9ed0-b7724291dca2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:58.397193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7b538f-7a44-4b93-9ed0-b7724291dca2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:58.419564 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9350268-0835-4637-8385-166f131ca802 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:58.454529 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe041b-c288-489d-96a4-fce19ef1faa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:58.455307 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe041b-c288-489d-96a4-fce19ef1faa6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:58.455584 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe041b-c288-489d-96a4-fce19ef1faa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:58.455846 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe041b-c288-489d-96a4-fce19ef1faa6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:58.466730 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe041b-c288-489d-96a4-fce19ef1faa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:48:58.484328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-8bbe041b-c288-489d-96a4-fce19ef1faa6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:48:58.484328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:58.484328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:58.484328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:58.484328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:58.484328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:58.484328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:58.484328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:58.484328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:58.484328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:58.484328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:58.484328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:58.484328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:48:58.484328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:48:58.484328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:48:58.484328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:48:58.485409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:48:58.485409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:48:58.485409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:48:58.485409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:48:58.485409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:58.485409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:58.485409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:58.485409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:58.485409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:58.485409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:58.485409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:58.485409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:58.485409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:58.485409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:58.485409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:58.485409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:58.486237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:58.486237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:58.486237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:58.486237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:58.486237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:48:58.486237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:48:58.486237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:48:58.486237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:48:58.486237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:48:58.486237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:48:58.486237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:48:58.486237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:58.486867 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 377/746] 10.222.0.44 () {60 vars in 1141 bytes} [Thu Jun 23 19:48:58 2022] POST /identity/v3/services => generated 240 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:48:58.486867 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b538f-7a44-4b93-9ed0-b7724291dca2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:58.487001 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 370/747] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:48:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:58.487001 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe041b-c288-489d-96a4-fce19ef1faa6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:58.528249 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:58.613528 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b538f-7a44-4b93-9ed0-b7724291dca2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:58.636323 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe041b-c288-489d-96a4-fce19ef1faa6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:58.660593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d510bc8-a407-4d1c-8557-f628b9a6a9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:58.661480 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d510bc8-a407-4d1c-8557-f628b9a6a9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:58.661762 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d510bc8-a407-4d1c-8557-f628b9a6a9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:58.662058 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d510bc8-a407-4d1c-8557-f628b9a6a9d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:58.662863 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d510bc8-a407-4d1c-8557-f628b9a6a9d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:58.663166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d510bc8-a407-4d1c-8557-f628b9a6a9d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:58.664761 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d510bc8-a407-4d1c-8557-f628b9a6a9d4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:58.677751 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 378/748] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:58.678383 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d510bc8-a407-4d1c-8557-f628b9a6a9d4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:58.681970 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:58.683710 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81898843-8733-46ea-b91b-3bc6c3d71c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:58.684651 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81898843-8733-46ea-b91b-3bc6c3d71c2d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:58.685136 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81898843-8733-46ea-b91b-3bc6c3d71c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:58.685412 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81898843-8733-46ea-b91b-3bc6c3d71c2d None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:58.686141 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81898843-8733-46ea-b91b-3bc6c3d71c2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:58.686445 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81898843-8733-46ea-b91b-3bc6c3d71c2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:58.688100 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81898843-8733-46ea-b91b-3bc6c3d71c2d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:58.701873 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 371/749] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:58.702493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81898843-8733-46ea-b91b-3bc6c3d71c2d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:58.795901 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d510bc8-a407-4d1c-8557-f628b9a6a9d4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:58.806606 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:58.830929 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81898843-8733-46ea-b91b-3bc6c3d71c2d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:58.838352 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23da08a-a575-43ea-8efe-405591a92868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:58.840057 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23da08a-a575-43ea-8efe-405591a92868 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:58.840600 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23da08a-a575-43ea-8efe-405591a92868 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:58.841084 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23da08a-a575-43ea-8efe-405591a92868 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:58.842435 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23da08a-a575-43ea-8efe-405591a92868 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:58.842970 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23da08a-a575-43ea-8efe-405591a92868 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:58.845123 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23da08a-a575-43ea-8efe-405591a92868 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:58.860387 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 379/750] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:48:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:58.861851 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23da08a-a575-43ea-8efe-405591a92868 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:58.873476 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d1aec-2518-4486-bbf7-213143d2a3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:58.874745 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d1aec-2518-4486-bbf7-213143d2a3a7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:58.875125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d1aec-2518-4486-bbf7-213143d2a3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:58.875493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d1aec-2518-4486-bbf7-213143d2a3a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:58.885236 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0d1aec-2518-4486-bbf7-213143d2a3a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:58.887148 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0d1aec-2518-4486-bbf7-213143d2a3a7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:58.905209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-8d0d1aec-2518-4486-bbf7-213143d2a3a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:58.905209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:58.905209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:58.905209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:58.905209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:58.905209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:58.905209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:58.905209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:58.905209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:58.905209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:58.905209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:58.905209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:58.905209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:48:58.905209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:48:58.905209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:48:58.906139 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:48:58.906139 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:58.906139 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:58.906139 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:58.906139 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:58.906139 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:58.906139 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:58.906139 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:58.906139 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:58.906139 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:58.906139 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:58.906139 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:58.906139 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:58.906139 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:58.906139 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:58.906139 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:58.906139 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:58.907328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:48:58.907328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:48:58.907328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:48:58.907328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:48:58.907328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:48:58.907328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:48:58.907328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:58.907328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:58.911934 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 372/751] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:48:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:58.912908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d1aec-2518-4486-bbf7-213143d2a3a7 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:58.949112 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:58.979403 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23da08a-a575-43ea-8efe-405591a92868 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:59.015813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bac27d-7258-4a80-b44f-ca66e0712c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:59.016731 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bac27d-7258-4a80-b44f-ca66e0712c9c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:59.017077 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bac27d-7258-4a80-b44f-ca66e0712c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:59.017413 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bac27d-7258-4a80-b44f-ca66e0712c9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:59.025593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bac27d-7258-4a80-b44f-ca66e0712c9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:59.028021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bac27d-7258-4a80-b44f-ca66e0712c9c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:59.046852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-37bac27d-7258-4a80-b44f-ca66e0712c9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:59.046852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:59.046852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:59.046852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:59.046852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:59.046852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:59.046852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:59.046852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:59.046852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:59.046852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:59.046852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:59.046852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:59.046852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:48:59.046852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:48:59.046852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:48:59.047773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:48:59.047773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:59.047773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:59.047773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:59.047773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:59.047773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:59.047773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:59.047773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:59.047773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:59.047773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:59.047773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:59.047773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:59.047773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:59.047773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:59.047773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:59.047773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:59.047773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:59.048712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:48:59.048712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:48:59.048712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:48:59.048712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:48:59.048712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:48:59.048712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:48:59.048712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:48:59.048712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:48:59.049151 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d1aec-2518-4486-bbf7-213143d2a3a7 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:59.056927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 380/752] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:48:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:59.057329 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bac27d-7258-4a80-b44f-ca66e0712c9c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:59.086984 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72dae1-6b34-4485-958c-f515dfa58a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:59.087827 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72dae1-6b34-4485-958c-f515dfa58a5f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:59.088105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72dae1-6b34-4485-958c-f515dfa58a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:59.088450 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e72dae1-6b34-4485-958c-f515dfa58a5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:59.099957 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e72dae1-6b34-4485-958c-f515dfa58a5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:59.101484 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e72dae1-6b34-4485-958c-f515dfa58a5f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:59.103508 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:59.120610 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-0e72dae1-6b34-4485-958c-f515dfa58a5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:59.120610 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:59.120610 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:59.120610 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:59.120610 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:59.120610 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:59.120610 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:59.120610 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:59.120610 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:59.120610 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:59.120610 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:59.120610 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:59.120610 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:48:59.120610 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:48:59.120610 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:48:59.121549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:48:59.121549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:59.121549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:59.121549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:59.121549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:59.121549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:59.121549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:59.121549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:59.121549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:59.121549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:59.121549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:59.121549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:59.121549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:59.121549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:59.121549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:59.121549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:59.121549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:59.122469 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:48:59.122469 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:48:59.122469 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:48:59.122469 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:59.126544 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 373/753] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:48:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:59.127373 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72dae1-6b34-4485-958c-f515dfa58a5f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:59.170290 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bac27d-7258-4a80-b44f-ca66e0712c9c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:59.204651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:59.208927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3fb5e-4c69-4b4d-9108-70d5d5885868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:59.209865 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3fb5e-4c69-4b4d-9108-70d5d5885868 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:59.210453 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3fb5e-4c69-4b4d-9108-70d5d5885868 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:59.210931 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3fb5e-4c69-4b4d-9108-70d5d5885868 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:59.212065 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad3fb5e-4c69-4b4d-9108-70d5d5885868 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:59.212609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad3fb5e-4c69-4b4d-9108-70d5d5885868 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:59.214329 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad3fb5e-4c69-4b4d-9108-70d5d5885868 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:59.233111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 381/754] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:48:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:59.234347 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3fb5e-4c69-4b4d-9108-70d5d5885868 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:59.247162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e72dae1-6b34-4485-958c-f515dfa58a5f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:59.282512 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df2d6c5-2130-464c-9cd6-650a7b0db1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:59.283182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2d6c5-2130-464c-9cd6-650a7b0db1ef None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:59.283487 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2d6c5-2130-464c-9cd6-650a7b0db1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:59.283951 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df2d6c5-2130-464c-9cd6-650a7b0db1ef None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:59.284588 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df2d6c5-2130-464c-9cd6-650a7b0db1ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:59.284893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df2d6c5-2130-464c-9cd6-650a7b0db1ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:59.286923 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df2d6c5-2130-464c-9cd6-650a7b0db1ef None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:59.305245 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 374/755] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:48:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:59.306121 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df2d6c5-2130-464c-9cd6-650a7b0db1ef None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:59.346789 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3fb5e-4c69-4b4d-9108-70d5d5885868 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:59.362710 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:59.385697 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152d362-7b53-485e-b21e-f899a287e0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:59.386688 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152d362-7b53-485e-b21e-f899a287e0b3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:59.387188 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152d362-7b53-485e-b21e-f899a287e0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:59.387684 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152d362-7b53-485e-b21e-f899a287e0b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:59.388656 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8152d362-7b53-485e-b21e-f899a287e0b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:59.389182 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8152d362-7b53-485e-b21e-f899a287e0b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:59.390829 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8152d362-7b53-485e-b21e-f899a287e0b3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:59.404154 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 382/756] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:48:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:59.406807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152d362-7b53-485e-b21e-f899a287e0b3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:59.414339 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df2d6c5-2130-464c-9cd6-650a7b0db1ef None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:59.450638 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007938d2-090e-48a5-b281-6414c7053e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:59.451479 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007938d2-090e-48a5-b281-6414c7053e39 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:59.451618 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007938d2-090e-48a5-b281-6414c7053e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:59.451872 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007938d2-090e-48a5-b281-6414c7053e39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:59.452784 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007938d2-090e-48a5-b281-6414c7053e39 None admin] RBAC: Adding query filter params (role.id=5f27c72acb19431fa3223c962bc42938, scope.project.id=155180ba63c446ca889a6eb3f94cc31a, user.id=1d37f369749143ce813c693adc5f2ec7) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:59.453093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007938d2-090e-48a5-b281-6414c7053e39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:59.454604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007938d2-090e-48a5-b281-6414c7053e39 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:59.464805 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 375/757] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:48:59 2022] GET /identity/v3/role_assignments?user.id=1d37f369749143ce813c693adc5f2ec7&scope.project.id=155180ba63c446ca889a6eb3f94cc31a&role.id=5f27c72acb19431fa3223c962bc42938 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:48:59.464805 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007938d2-090e-48a5-b281-6414c7053e39 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:59.474957 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:59.582241 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152d362-7b53-485e-b21e-f899a287e0b3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:59.655687 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee77d55e-9ed4-467d-931f-d948fb39bd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:59.656966 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77d55e-9ed4-467d-931f-d948fb39bd8b None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:59.657485 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77d55e-9ed4-467d-931f-d948fb39bd8b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:59.658101 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77d55e-9ed4-467d-931f-d948fb39bd8b None admin] PATH_INFO: `/v3/projects/155180ba63c446ca889a6eb3f94cc31a/users/a63d1d902c024c97a38f7d92f73814d3/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:59.660805 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee77d55e-9ed4-467d-931f-d948fb39bd8b None admin] RBAC: Authorizing `identity:create_grant(project_id=155180ba63c446ca889a6eb3f94cc31a, user_id=a63d1d902c024c97a38f7d92f73814d3, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:59.662974 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee77d55e-9ed4-467d-931f-d948fb39bd8b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:59.665753 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007938d2-090e-48a5-b281-6414c7053e39 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:59.692681 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:48:59.703037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8698b1-c4b9-4d08-8133-3f00f6e60cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:59.703898 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8698b1-c4b9-4d08-8133-3f00f6e60cda None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:59.704193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8698b1-c4b9-4d08-8133-3f00f6e60cda None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:59.704518 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8698b1-c4b9-4d08-8133-3f00f6e60cda None admin] PATH_INFO: `/v3/users/swift` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:59.713679 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8698b1-c4b9-4d08-8133-3f00f6e60cda None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:59.715257 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8698b1-c4b9-4d08-8133-3f00f6e60cda None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:59.731212 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-8d8698b1-c4b9-4d08-8133-3f00f6e60cda None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 19:48:59.731212 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:48:59.731212 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:48:59.731212 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:48:59.731212 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:48:59.731212 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:48:59.731212 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:48:59.731212 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:48:59.731212 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:48:59.731212 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:48:59.731212 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:48:59.731212 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:48:59.731212 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:48:59.731212 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:48:59.731212 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:48:59.731212 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:48:59.732220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:48:59.732220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:48:59.732220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:48:59.732220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:59.732220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:48:59.732220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:48:59.732220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:48:59.732220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:48:59.732220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:48:59.732220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:48:59.732220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:48:59.732220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:48:59.732220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:48:59.732220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:48:59.732220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:48:59.732220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:48:59.732220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:48:59.732220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:48:59.733217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:48:59.733217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:48:59.733217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:48:59.733217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:48:59.733217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:48:59.733217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:48:59.733217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:48:59.733217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:48:59.733217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 19:48:59.733217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:48:59.733217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 376/758] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:48:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:48:59.734431 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bd04e-dcf9-4b04-b200-582c0ad0c5dc None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:59.734431 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bd04e-dcf9-4b04-b200-582c0ad0c5dc None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:59.734431 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bd04e-dcf9-4b04-b200-582c0ad0c5dc None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:59.740733 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 377/759] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:48:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:48:59.744599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bd04e-dcf9-4b04-b200-582c0ad0c5dc None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:59.745505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 383/760] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jun 23 19:48:59 2022] PUT /identity/v3/projects/155180ba63c446ca889a6eb3f94cc31a/users/a63d1d902c024c97a38f7d92f73814d3/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:48:59.746909 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085d797-0d5e-4f28-af5a-da2d15b80ba6 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:59.747202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085d797-0d5e-4f28-af5a-da2d15b80ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:59.747529 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b085d797-0d5e-4f28-af5a-da2d15b80ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:59.780152 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b085d797-0d5e-4f28-af5a-da2d15b80ba6 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:48:59.781992 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b085d797-0d5e-4f28-af5a-da2d15b80ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:48:59.928398 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 384/761] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:48:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:48:59.936347 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bd04e-dcf9-4b04-b200-582c0ad0c5dc None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:48:59.949924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085d797-0d5e-4f28-af5a-da2d15b80ba6 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:48:59.984413 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d695ec-426c-4832-a6c1-a415eb1efeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:48:59.985295 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d695ec-426c-4832-a6c1-a415eb1efeed None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:48:59.985408 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d695ec-426c-4832-a6c1-a415eb1efeed None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:48:59.985631 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d695ec-426c-4832-a6c1-a415eb1efeed None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:48:59.986471 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d695ec-426c-4832-a6c1-a415eb1efeed None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:48:59.986790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d695ec-426c-4832-a6c1-a415eb1efeed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:48:59.988287 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d695ec-426c-4832-a6c1-a415eb1efeed None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:48:59.993214 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:00.018987 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 378/762] 10.222.0.44 () {56 vars in 1135 bytes} [Thu Jun 23 19:48:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:00.024609 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d695ec-426c-4832-a6c1-a415eb1efeed None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:00.063420 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:00.074245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b085d797-0d5e-4f28-af5a-da2d15b80ba6 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:00.112530 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444be45-f563-4251-929b-c0dfeb10ff55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:00.113236 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444be45-f563-4251-929b-c0dfeb10ff55 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:00.113666 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444be45-f563-4251-929b-c0dfeb10ff55 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:00.114486 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444be45-f563-4251-929b-c0dfeb10ff55 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:00.115219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444be45-f563-4251-929b-c0dfeb10ff55 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:00.116783 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444be45-f563-4251-929b-c0dfeb10ff55 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:00.136344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d695ec-426c-4832-a6c1-a415eb1efeed None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:00.142648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-a444be45-f563-4251-929b-c0dfeb10ff55 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 19:49:00.142648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:00.142648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 19:49:00.142648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 19:49:00.142648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 19:49:00.142648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 19:49:00.142648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 19:49:00.142648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:00.142648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 19:49:00.142648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:00.142648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:49:00.142648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:00.142648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:00.142648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:00.142648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:00.142648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:00.143594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:00.143594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:00.143594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:00.143594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:00.143594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:00.143594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:00.143594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:00.143594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 19:49:00.143594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 19:49:00.143594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 19:49:00.143594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 19:49:00.143594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:00.143594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:00.143594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:00.143594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:00.143594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:00.144466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:00.144466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:00.144466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:00.144466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:00.144466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:00.144466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:00.144466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:00.144466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:00.144466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:00.144466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:00.144466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:00.144466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 19:49:00.144466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:00.144466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 19:49:00.144466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:00.145309 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 385/763] 10.222.0.44 () {56 vars in 1129 bytes} [Thu Jun 23 19:48:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:00.149691 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444be45-f563-4251-929b-c0dfeb10ff55 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:00.176778 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b0a37-be69-400a-ae64-4544d53d48b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:00.177791 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b0a37-be69-400a-ae64-4544d53d48b8 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:00.178288 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b0a37-be69-400a-ae64-4544d53d48b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:00.178801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b0a37-be69-400a-ae64-4544d53d48b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:00.310229 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37b0a37-be69-400a-ae64-4544d53d48b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:00.312013 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37b0a37-be69-400a-ae64-4544d53d48b8 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:00.357628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:00.362715 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444be45-f563-4251-929b-c0dfeb10ff55 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:00.379740 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:00.424081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea9d47-b96f-47ae-b3a0-36ae700b91f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:00.424778 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea9d47-b96f-47ae-b3a0-36ae700b91f6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:00.425065 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea9d47-b96f-47ae-b3a0-36ae700b91f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:00.425341 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea9d47-b96f-47ae-b3a0-36ae700b91f6 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:00.426052 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ea9d47-b96f-47ae-b3a0-36ae700b91f6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:00.426899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ea9d47-b96f-47ae-b3a0-36ae700b91f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:00.428342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ea9d47-b96f-47ae-b3a0-36ae700b91f6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:00.435945 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 386/764] 10.222.0.44 () {56 vars in 1124 bytes} [Thu Jun 23 19:49:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:00.442684 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea9d47-b96f-47ae-b3a0-36ae700b91f6 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:00.492936 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 379/765] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:49:00 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 19:49:00.499512 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b0a37-be69-400a-ae64-4544d53d48b8 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:00.625553 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea9d47-b96f-47ae-b3a0-36ae700b91f6 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:00.667109 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b0a37-be69-400a-ae64-4544d53d48b8 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:00.688160 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25160f2-e30b-4ad8-a3ab-99380aa2b93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:00.688826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25160f2-e30b-4ad8-a3ab-99380aa2b93b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:00.689109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25160f2-e30b-4ad8-a3ab-99380aa2b93b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:00.689433 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25160f2-e30b-4ad8-a3ab-99380aa2b93b None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:00.690153 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25160f2-e30b-4ad8-a3ab-99380aa2b93b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:00.690463 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25160f2-e30b-4ad8-a3ab-99380aa2b93b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:00.691840 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25160f2-e30b-4ad8-a3ab-99380aa2b93b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:00.704436 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 387/766] 10.222.0.44 () {56 vars in 1124 bytes} [Thu Jun 23 19:49:00 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:00.708492 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25160f2-e30b-4ad8-a3ab-99380aa2b93b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:00.727214 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae293d9-cf0b-4c81-a514-32b400326bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:00.728122 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae293d9-cf0b-4c81-a514-32b400326bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:00.728447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae293d9-cf0b-4c81-a514-32b400326bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:00.728832 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae293d9-cf0b-4c81-a514-32b400326bb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:00.732018 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:00.740403 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae293d9-cf0b-4c81-a514-32b400326bb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:00.740971 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:00.743941 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae293d9-cf0b-4c81-a514-32b400326bb1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:00.761090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-bae293d9-cf0b-4c81-a514-32b400326bb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:00.761090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:00.761090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:00.761090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:00.761090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:00.761090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:00.761090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:00.761090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:00.761090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:00.761090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:00.761090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:00.761090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:00.761090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:49:00.761090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:49:00.761090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:49:00.762077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:49:00.762077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:00.762077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:00.762077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:00.762077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:00.762077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:00.762077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:00.762077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:00.762077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:00.762077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:00.762077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:00.762077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:00.762077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:00.762077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:00.762077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:00.762077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:00.762077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:00.762963 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:49:00.762963 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:49:00.762963 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:00.762963 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:00.762963 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 380/767] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:00.764369 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178aa892-d219-40f2-8312-2cc938a720f3 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:00.764670 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178aa892-d219-40f2-8312-2cc938a720f3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:00.765040 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178aa892-d219-40f2-8312-2cc938a720f3 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:00.770714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 381/768] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:00.771611 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178aa892-d219-40f2-8312-2cc938a720f3 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:00.832607 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25160f2-e30b-4ad8-a3ab-99380aa2b93b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:00.883993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178aa892-d219-40f2-8312-2cc938a720f3 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:00.884792 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28127b9-f9f9-41b9-961e-38779f2af8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:00.885758 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28127b9-f9f9-41b9-961e-38779f2af8ff None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:00.886051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28127b9-f9f9-41b9-961e-38779f2af8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:00.886327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28127b9-f9f9-41b9-961e-38779f2af8ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:00.888057 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28127b9-f9f9-41b9-961e-38779f2af8ff None admin] RBAC: Adding query filter params (interface=public, service_id=dfd1ae99fc034def9f6800562fb36ce6, region_id=RegionOne) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:00.888446 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28127b9-f9f9-41b9-961e-38779f2af8ff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:00.889925 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28127b9-f9f9-41b9-961e-38779f2af8ff None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:00.902283 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 388/769] 10.222.0.44 () {56 vars in 1251 bytes} [Thu Jun 23 19:49:00 2022] GET /identity/v3/endpoints?service_id=dfd1ae99fc034def9f6800562fb36ce6&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:00.903997 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a14fac-c17a-4301-a3de-dfd7fd9451ab None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:00.904289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a14fac-c17a-4301-a3de-dfd7fd9451ab None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:00.904588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a14fac-c17a-4301-a3de-dfd7fd9451ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:00.921877 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ceb3d-3d4b-4ddb-9212-777830b1a708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:00.922622 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ceb3d-3d4b-4ddb-9212-777830b1a708 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:00.922969 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ceb3d-3d4b-4ddb-9212-777830b1a708 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:00.923330 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ceb3d-3d4b-4ddb-9212-777830b1a708 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:00.924129 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ceb3d-3d4b-4ddb-9212-777830b1a708 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:00.924547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ceb3d-3d4b-4ddb-9212-777830b1a708 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:00.925792 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-75a14fac-c17a-4301-a3de-dfd7fd9451ab None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:00.926049 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ceb3d-3d4b-4ddb-9212-777830b1a708 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:00.927652 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-75a14fac-c17a-4301-a3de-dfd7fd9451ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:00.939665 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 382/770] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:49:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:00.940683 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ceb3d-3d4b-4ddb-9212-777830b1a708 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:00.992900 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:01.017858 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 389/771] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:01.018712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a14fac-c17a-4301-a3de-dfd7fd9451ab None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:01.044617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:01.071740 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ceb3d-3d4b-4ddb-9212-777830b1a708 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:01.116143 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b659e-d93f-4df2-a74c-f78abeb9ca6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:01.116841 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b659e-d93f-4df2-a74c-f78abeb9ca6b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:01.117119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b659e-d93f-4df2-a74c-f78abeb9ca6b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:01.117391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4b659e-d93f-4df2-a74c-f78abeb9ca6b None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:01.118075 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4b659e-d93f-4df2-a74c-f78abeb9ca6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:01.119639 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4b659e-d93f-4df2-a74c-f78abeb9ca6b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:01.125285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 383/772] 10.222.0.44 () {56 vars in 1087 bytes} [Thu Jun 23 19:49:00 2022] GET /identity/v3/services => generated 772 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:01.126172 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b659e-d93f-4df2-a74c-f78abeb9ca6b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:01.149906 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a14fac-c17a-4301-a3de-dfd7fd9451ab None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:01.201574 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0e17e-2b25-4df1-af04-a72744d1856e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:01.202983 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0e17e-2b25-4df1-af04-a72744d1856e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:01.203275 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0e17e-2b25-4df1-af04-a72744d1856e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:01.203553 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0e17e-2b25-4df1-af04-a72744d1856e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:01.214098 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0e17e-2b25-4df1-af04-a72744d1856e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:01.216203 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d0e17e-2b25-4df1-af04-a72744d1856e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:01.241019 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-76d0e17e-2b25-4df1-af04-a72744d1856e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:49:01.241019 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:01.241019 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:01.241019 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:01.241019 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:01.241019 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:01.241019 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:01.241019 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:01.241019 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:01.241019 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:01.241019 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:01.241019 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:01.241019 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:49:01.241019 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:49:01.241019 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:49:01.241930 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:49:01.241930 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:01.241930 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:01.241930 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:01.241930 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:01.241930 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:01.241930 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:01.241930 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:01.241930 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:01.241930 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:01.241930 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:01.241930 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:01.241930 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:01.241930 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:01.241930 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:01.241930 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:01.241930 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:01.242835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:49:01.242835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:49:01.242835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:49:01.242835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:49:01.242835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:49:01.242835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:49:01.242835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:49:01.242835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:01.242835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 390/773] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:49:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:01.256369 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0e17e-2b25-4df1-af04-a72744d1856e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:01.265079 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4b659e-d93f-4df2-a74c-f78abeb9ca6b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:01.295540 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:01.350533 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0aa997-46d2-47f3-8b6b-dcdebd5d9fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:01.351397 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0aa997-46d2-47f3-8b6b-dcdebd5d9fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:01.351683 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0aa997-46d2-47f3-8b6b-dcdebd5d9fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:01.351962 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0aa997-46d2-47f3-8b6b-dcdebd5d9fe7 None admin] PATH_INFO: `/v3/services/network` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:01.352681 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0aa997-46d2-47f3-8b6b-dcdebd5d9fe7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:01.354091 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0aa997-46d2-47f3-8b6b-dcdebd5d9fe7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:01.367746 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-fe0aa997-46d2-47f3-8b6b-dcdebd5d9fe7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 19:49:01.367746 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:01.367746 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 19:49:01.367746 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 19:49:01.367746 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 19:49:01.367746 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 19:49:01.367746 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 19:49:01.367746 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:01.367746 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 19:49:01.367746 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:01.367746 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:49:01.367746 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:01.367746 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:01.367746 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:01.367746 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:01.367746 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:01.367746 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:01.368798 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:01.368798 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:01.368798 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:01.368798 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:01.368798 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:01.368798 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:01.368798 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 19:49:01.368798 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 19:49:01.368798 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 19:49:01.368798 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 19:49:01.368798 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:01.368798 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:01.368798 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:01.368798 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:01.368798 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:01.368798 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:01.369643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:01.369643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:01.369643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:01.369643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:01.369643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:01.369643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:01.369643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:01.369643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:01.369643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:01.369643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:01.369643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 19:49:01.369643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:01.369643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 19:49:01.369643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:01.369643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 384/774] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:49:01 2022] GET /identity/v3/services/network => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:01.377305 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0aa997-46d2-47f3-8b6b-dcdebd5d9fe7 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:01.395407 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:01.427259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0e17e-2b25-4df1-af04-a72744d1856e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:01.506970 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8b58f-f34d-4838-b84d-f1afc7c5f316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:01.509096 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8b58f-f34d-4838-b84d-f1afc7c5f316 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:01.509389 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8b58f-f34d-4838-b84d-f1afc7c5f316 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:01.509668 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8b58f-f34d-4838-b84d-f1afc7c5f316 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:01.510465 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c8b58f-f34d-4838-b84d-f1afc7c5f316 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:01.510783 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c8b58f-f34d-4838-b84d-f1afc7c5f316 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:01.512229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c8b58f-f34d-4838-b84d-f1afc7c5f316 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:01.523708 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0aa997-46d2-47f3-8b6b-dcdebd5d9fe7 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:01.530708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 391/775] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:49:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:01.536967 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8b58f-f34d-4838-b84d-f1afc7c5f316 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:01.564759 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5086b84-8d02-48be-a0f4-60d79848843e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:01.565757 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5086b84-8d02-48be-a0f4-60d79848843e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:01.566223 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5086b84-8d02-48be-a0f4-60d79848843e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:01.566685 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5086b84-8d02-48be-a0f4-60d79848843e None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:01.567600 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5086b84-8d02-48be-a0f4-60d79848843e None admin] RBAC: Adding query filter params (name=network) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:01.568142 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5086b84-8d02-48be-a0f4-60d79848843e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:01.570030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5086b84-8d02-48be-a0f4-60d79848843e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:01.579586 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 385/776] 10.222.0.44 () {56 vars in 1112 bytes} [Thu Jun 23 19:49:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:01.584379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5086b84-8d02-48be-a0f4-60d79848843e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:01.679773 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:01.693364 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8b58f-f34d-4838-b84d-f1afc7c5f316 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:01.753057 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914e4d0-38a2-4160-9e87-b009e940be14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:01.753720 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914e4d0-38a2-4160-9e87-b009e940be14 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:01.754003 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914e4d0-38a2-4160-9e87-b009e940be14 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:01.754279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914e4d0-38a2-4160-9e87-b009e940be14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:01.755157 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914e4d0-38a2-4160-9e87-b009e940be14 None admin] RBAC: Adding query filter params (role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb, scope.project.id=155180ba63c446ca889a6eb3f94cc31a, user.id=96db03079816461bb322633f64b9fb9d) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:01.755469 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914e4d0-38a2-4160-9e87-b009e940be14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:01.756959 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914e4d0-38a2-4160-9e87-b009e940be14 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:01.768818 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 392/777] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:49:01 2022] GET /identity/v3/role_assignments?user.id=96db03079816461bb322633f64b9fb9d&scope.project.id=155180ba63c446ca889a6eb3f94cc31a&role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:01.784976 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5086b84-8d02-48be-a0f4-60d79848843e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:01.823540 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dfd14a-6bc2-463e-88ea-a89581d9ea42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:01.824218 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dfd14a-6bc2-463e-88ea-a89581d9ea42 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:01.824530 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dfd14a-6bc2-463e-88ea-a89581d9ea42 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:01.824806 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dfd14a-6bc2-463e-88ea-a89581d9ea42 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:01.825534 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dfd14a-6bc2-463e-88ea-a89581d9ea42 None admin] RBAC: Adding query filter params (type=network) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:01.825847 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dfd14a-6bc2-463e-88ea-a89581d9ea42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:01.827257 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dfd14a-6bc2-463e-88ea-a89581d9ea42 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:01.833932 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 386/778] 10.222.0.44 () {56 vars in 1112 bytes} [Thu Jun 23 19:49:01 2022] GET /identity/v3/services?type=network => generated 350 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:01.839373 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914e4d0-38a2-4160-9e87-b009e940be14 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:01.859861 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:01.899636 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d0eb1-f055-48b0-b5e2-48b299a4c7f5 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:01.900433 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d0eb1-f055-48b0-b5e2-48b299a4c7f5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:01.900938 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62d0eb1-f055-48b0-b5e2-48b299a4c7f5 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:01.902560 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 387/779] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:49:01.908532 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb8e8f-78cb-4747-8051-2e2a9ccae2aa None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:01.909176 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb8e8f-78cb-4747-8051-2e2a9ccae2aa None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:01.909676 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb8e8f-78cb-4747-8051-2e2a9ccae2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:01.931977 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-0beb8e8f-78cb-4747-8051-2e2a9ccae2aa None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:01.933617 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0beb8e8f-78cb-4747-8051-2e2a9ccae2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:02.041251 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914e4d0-38a2-4160-9e87-b009e940be14 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:02.051507 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 388/780] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 19:49:02.089194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71a1c0-efcf-4929-882b-e015ef62fdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:02.089855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71a1c0-efcf-4929-882b-e015ef62fdb9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:02.090138 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71a1c0-efcf-4929-882b-e015ef62fdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:02.090408 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71a1c0-efcf-4929-882b-e015ef62fdb9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:02.091200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe71a1c0-efcf-4929-882b-e015ef62fdb9 None admin] RBAC: Adding query filter params (interface=public, service_id=ed67c3bb3f384ab9a7e7c50e195293bc, region_id=RegionOne) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:02.091512 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe71a1c0-efcf-4929-882b-e015ef62fdb9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:02.092925 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe71a1c0-efcf-4929-882b-e015ef62fdb9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:02.101114 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb8e8f-78cb-4747-8051-2e2a9ccae2aa None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:02.117884 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 393/781] 10.222.0.44 () {56 vars in 1251 bytes} [Thu Jun 23 19:49:01 2022] GET /identity/v3/endpoints?service_id=ed67c3bb3f384ab9a7e7c50e195293bc&interface=public®ion_id=RegionOne => generated 194 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:02.122788 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71a1c0-efcf-4929-882b-e015ef62fdb9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:02.162473 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:02.240793 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:02.272730 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71a1c0-efcf-4929-882b-e015ef62fdb9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:02.283019 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb8e8f-78cb-4747-8051-2e2a9ccae2aa None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:02.319191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba84ca5-bbe1-491a-9721-57b7abf9f6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:02.319998 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba84ca5-bbe1-491a-9721-57b7abf9f6c3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:02.320282 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba84ca5-bbe1-491a-9721-57b7abf9f6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:02.320563 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487019d8-740e-4cf2-9410-937ada0bf716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:02.320685 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba84ca5-bbe1-491a-9721-57b7abf9f6c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:02.321800 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487019d8-740e-4cf2-9410-937ada0bf716 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:02.322289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487019d8-740e-4cf2-9410-937ada0bf716 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:02.322872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487019d8-740e-4cf2-9410-937ada0bf716 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:02.323860 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487019d8-740e-4cf2-9410-937ada0bf716 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:02.325474 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487019d8-740e-4cf2-9410-937ada0bf716 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:02.331378 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba84ca5-bbe1-491a-9721-57b7abf9f6c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:49:02.331847 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 394/782] 10.222.0.44 () {56 vars in 1087 bytes} [Thu Jun 23 19:49:02 2022] GET /identity/v3/services => generated 772 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:02.333959 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c80e13-bdca-4475-a3cb-f3de0e739c61 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:02.334468 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c80e13-bdca-4475-a3cb-f3de0e739c61 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:02.334949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c80e13-bdca-4475-a3cb-f3de0e739c61 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:02.336826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 395/783] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:02.344381 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-5ba84ca5-bbe1-491a-9721-57b7abf9f6c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:49:02.344381 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:02.344381 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:02.344381 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:02.344381 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:02.344381 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:02.344381 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:02.344381 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:02.344381 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:02.344381 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:02.344381 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:02.344381 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:02.344381 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:49:02.344381 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:49:02.344381 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:49:02.344381 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:49:02.345409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:49:02.345409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:49:02.345409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:49:02.345409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:49:02.345409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:02.345409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:02.345409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:02.345409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:02.345409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:02.345409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:02.345409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:02.345409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:02.345409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:02.345409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:02.345409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:02.345409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:02.346257 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:02.346257 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:02.346257 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:02.346257 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:02.346257 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:49:02.346257 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:49:02.346257 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:49:02.346257 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:49:02.346257 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:49:02.346257 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:49:02.346257 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:49:02.346257 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:02.346896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 389/784] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jun 23 19:49:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:49:02.348703 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba84ca5-bbe1-491a-9721-57b7abf9f6c3 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:02.349129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66d0e8-3fa2-46c7-9d03-9c9875a32ae9 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:02.349691 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66d0e8-3fa2-46c7-9d03-9c9875a32ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:02.350122 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66d0e8-3fa2-46c7-9d03-9c9875a32ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:02.379128 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ad66d0e8-3fa2-46c7-9d03-9c9875a32ae9 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:02.381291 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ad66d0e8-3fa2-46c7-9d03-9c9875a32ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:02.463685 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba84ca5-bbe1-491a-9721-57b7abf9f6c3 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:02.465640 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:02.503994 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d84d2-fefa-4608-914c-d4392e91853d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:02.505259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d84d2-fefa-4608-914c-d4392e91853d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:02.505595 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d84d2-fefa-4608-914c-d4392e91853d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:02.505788 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d84d2-fefa-4608-914c-d4392e91853d None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:02.507097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d84d2-fefa-4608-914c-d4392e91853d None admin] RBAC: Adding query filter params (name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:02.507097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d84d2-fefa-4608-914c-d4392e91853d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:02.508282 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d84d2-fefa-4608-914c-d4392e91853d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:02.522229 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 390/785] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jun 23 19:49:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:02.523678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d84d2-fefa-4608-914c-d4392e91853d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:02.542236 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 396/786] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:02.566038 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66d0e8-3fa2-46c7-9d03-9c9875a32ae9 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:02.631319 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:02.633278 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d84d2-fefa-4608-914c-d4392e91853d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:02.668963 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e516c-92d9-41d9-8018-7a753db72e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:02.669766 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e516c-92d9-41d9-8018-7a753db72e06 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:02.670046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e516c-92d9-41d9-8018-7a753db72e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:02.670318 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e516c-92d9-41d9-8018-7a753db72e06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:02.672113 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66d0e8-3fa2-46c7-9d03-9c9875a32ae9 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:02.685947 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e516c-92d9-41d9-8018-7a753db72e06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:02.685947 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e516c-92d9-41d9-8018-7a753db72e06 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:02.712350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-107e516c-92d9-41d9-8018-7a753db72e06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:02.712350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:02.712350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:02.712350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:02.712350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:02.712350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:02.712350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:02.712350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:02.712350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:02.712350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:02.712350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:02.712350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:02.712350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:49:02.712350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:49:02.712350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:49:02.713331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:49:02.713331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:02.713331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:02.713331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:02.713331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:02.713331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:02.713331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:02.713331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:02.713331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:02.713331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:02.713331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:02.713331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:02.713331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:02.713331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:02.713331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:02.713331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:02.713331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:02.714233 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:49:02.714233 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:49:02.714233 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:02.714233 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:02.721202 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 391/787] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:02.725252 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e516c-92d9-41d9-8018-7a753db72e06 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:02.746626 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cda23d-9672-4cb2-94cb-d1617ac29b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:02.747485 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cda23d-9672-4cb2-94cb-d1617ac29b68 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:02.747772 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cda23d-9672-4cb2-94cb-d1617ac29b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:02.748047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cda23d-9672-4cb2-94cb-d1617ac29b68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:02.750431 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:02.771639 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cda23d-9672-4cb2-94cb-d1617ac29b68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:49:02.785189 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-e2cda23d-9672-4cb2-94cb-d1617ac29b68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:49:02.785189 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:02.785189 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:02.785189 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:02.785189 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:02.785189 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:02.785189 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:02.785189 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:02.785189 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:02.785189 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:02.785189 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:02.785189 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:02.785189 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:49:02.785189 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:49:02.785189 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:49:02.785189 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:49:02.786159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:49:02.786159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:49:02.786159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:49:02.786159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:49:02.786159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:02.786159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:02.786159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:02.786159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:02.786159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:02.786159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:02.786159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:02.786159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:02.786159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:02.786159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:02.786159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:02.786159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:02.787006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:02.787006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:02.787006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:02.787006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:02.787006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:49:02.787006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:49:02.787006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:49:02.787006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:49:02.787006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:49:02.787006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:49:02.787006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:49:02.787006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:02.790527 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 397/788] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:49:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:49:02.796104 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cda23d-9672-4cb2-94cb-d1617ac29b68 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:02.907778 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e516c-92d9-41d9-8018-7a753db72e06 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:02.958810 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cda23d-9672-4cb2-94cb-d1617ac29b68 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:02.979143 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec947a-a02f-4cc9-b9bf-e1564fd132ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:02.979815 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec947a-a02f-4cc9-b9bf-e1564fd132ca None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:02.980102 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec947a-a02f-4cc9-b9bf-e1564fd132ca None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:02.980400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec947a-a02f-4cc9-b9bf-e1564fd132ca None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:02.981195 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec947a-a02f-4cc9-b9bf-e1564fd132ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:02.981504 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec947a-a02f-4cc9-b9bf-e1564fd132ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:02.982948 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec947a-a02f-4cc9-b9bf-e1564fd132ca None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:02.985758 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:02.996860 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e8178-c738-43e7-8e3d-c9963411246c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:02.997845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e8178-c738-43e7-8e3d-c9963411246c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:02.998338 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e8178-c738-43e7-8e3d-c9963411246c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:02.998806 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e8178-c738-43e7-8e3d-c9963411246c None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:02.999743 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e8178-c738-43e7-8e3d-c9963411246c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:03.000253 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e8178-c738-43e7-8e3d-c9963411246c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:03.001844 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e8178-c738-43e7-8e3d-c9963411246c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:03.008772 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 392/789] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:49:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:03.016758 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 398/790] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:49:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:03.016872 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec947a-a02f-4cc9-b9bf-e1564fd132ca None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:03.018477 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e8178-c738-43e7-8e3d-c9963411246c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:03.117509 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:03.138667 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec947a-a02f-4cc9-b9bf-e1564fd132ca None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:03.211018 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e8178-c738-43e7-8e3d-c9963411246c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:03.233037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a36cc-ad7d-4dfa-9972-af4cb418a47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:03.233874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a36cc-ad7d-4dfa-9972-af4cb418a47e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:03.234158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a36cc-ad7d-4dfa-9972-af4cb418a47e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:03.234431 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a36cc-ad7d-4dfa-9972-af4cb418a47e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:03.259192 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3df4f-ea08-476a-99c4-2c59280088aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:03.259993 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3df4f-ea08-476a-99c4-2c59280088aa None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:03.260283 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3df4f-ea08-476a-99c4-2c59280088aa None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:03.260588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3df4f-ea08-476a-99c4-2c59280088aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:03.261477 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a36cc-ad7d-4dfa-9972-af4cb418a47e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:03.262900 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a36cc-ad7d-4dfa-9972-af4cb418a47e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:03.271356 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3df4f-ea08-476a-99c4-2c59280088aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:03.272871 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3df4f-ea08-476a-99c4-2c59280088aa None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:03.292260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-56f3df4f-ea08-476a-99c4-2c59280088aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:03.292260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:03.292260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:03.292260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:03.292260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:03.292260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:03.292260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:03.292260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:03.292260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:03.292260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:03.292260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:03.292260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:03.292260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:49:03.292260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:49:03.292260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:49:03.293220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:49:03.293220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:03.293220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:03.293220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:03.293220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:03.293220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:03.293220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:03.293220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:03.293220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:03.293220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:03.293220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:03.293220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:03.293220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:03.293220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:03.293220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:03.293220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:03.293220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:03.294115 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:49:03.294115 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:49:03.294115 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:03.294115 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:03.294115 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 399/791] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:03.305666 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3df4f-ea08-476a-99c4-2c59280088aa None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:03.306553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-841a36cc-ad7d-4dfa-9972-af4cb418a47e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 19:49:03.306553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:03.306553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:03.306553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:03.306553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:03.306553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:03.306553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:03.306553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:03.306553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:03.306553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:03.306553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:03.306553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:03.306553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:49:03.306553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:49:03.306553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:49:03.306553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:49:03.307456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:03.307456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:03.307456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:49:03.307456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:49:03.307456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:49:03.307456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:49:03.307456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:03.307456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:03.307456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:03.307456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:03.307456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:03.307456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:03.307456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:03.307456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:03.307456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:03.307456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:03.307456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:03.307456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:03.308428 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:03.308428 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:03.308428 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:49:03.308428 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:49:03.308428 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:49:03.308428 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:49:03.308428 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:49:03.308428 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:49:03.308428 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 19:49:03.308428 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:03.308428 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 393/792] 10.222.0.44 () {56 vars in 1117 bytes} [Thu Jun 23 19:49:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:03.317964 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a36cc-ad7d-4dfa-9972-af4cb418a47e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:03.368640 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:03.456160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a36cc-ad7d-4dfa-9972-af4cb418a47e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:03.485027 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:03.500086 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac14621-d9bf-4986-8341-35a2871e35d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:03.502393 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3df4f-ea08-476a-99c4-2c59280088aa None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:03.503282 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac14621-d9bf-4986-8341-35a2871e35d9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:03.503758 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac14621-d9bf-4986-8341-35a2871e35d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:03.504232 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac14621-d9bf-4986-8341-35a2871e35d9 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:03.505369 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac14621-d9bf-4986-8341-35a2871e35d9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:03.506662 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac14621-d9bf-4986-8341-35a2871e35d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:03.508906 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac14621-d9bf-4986-8341-35a2871e35d9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:03.548674 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 394/793] 10.222.0.44 () {56 vars in 1149 bytes} [Thu Jun 23 19:49:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:03.557243 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac14621-d9bf-4986-8341-35a2871e35d9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:03.561721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba416488-ae26-4229-819e-9b4083bc983d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:03.562997 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba416488-ae26-4229-819e-9b4083bc983d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:03.563506 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba416488-ae26-4229-819e-9b4083bc983d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:03.563976 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba416488-ae26-4229-819e-9b4083bc983d None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:03.564923 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba416488-ae26-4229-819e-9b4083bc983d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:03.565432 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba416488-ae26-4229-819e-9b4083bc983d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:03.566997 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba416488-ae26-4229-819e-9b4083bc983d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:03.580211 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 400/794] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:49:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:03.585768 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba416488-ae26-4229-819e-9b4083bc983d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:03.684336 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:03.686952 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac14621-d9bf-4986-8341-35a2871e35d9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:03.688662 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba416488-ae26-4229-819e-9b4083bc983d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:03.724051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5cb3c4-6264-4db7-a806-8aee1ff947fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:03.724332 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554bc370-bc78-4f46-88ed-2cbf9d23beca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:03.724908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cb3c4-6264-4db7-a806-8aee1ff947fe None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:03.725028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bc370-bc78-4f46-88ed-2cbf9d23beca None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:03.725278 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cb3c4-6264-4db7-a806-8aee1ff947fe None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:03.725392 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bc370-bc78-4f46-88ed-2cbf9d23beca None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:03.725844 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5cb3c4-6264-4db7-a806-8aee1ff947fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:03.726074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bc370-bc78-4f46-88ed-2cbf9d23beca None admin] PATH_INFO: `/v3/users/nova` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:03.735135 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554bc370-bc78-4f46-88ed-2cbf9d23beca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:03.736642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554bc370-bc78-4f46-88ed-2cbf9d23beca None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:03.738545 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5cb3c4-6264-4db7-a806-8aee1ff947fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:03.740188 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5cb3c4-6264-4db7-a806-8aee1ff947fe None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:03.752273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-554bc370-bc78-4f46-88ed-2cbf9d23beca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 19:49:03.752273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:03.752273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:03.752273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:03.752273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:03.752273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:03.752273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:03.752273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:03.752273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:03.752273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:03.752273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:03.752273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:03.752273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:49:03.752273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:49:03.752273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:49:03.752273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:49:03.753278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:03.753278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:03.753278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:49:03.753278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:49:03.753278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:49:03.753278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:49:03.753278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:03.753278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:03.753278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:03.753278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:03.753278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:03.753278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:03.753278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:03.753278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:03.753278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:03.753278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:03.753278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:03.753278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:03.754224 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:03.754224 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:03.754224 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:49:03.754224 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:49:03.754224 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:49:03.754224 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:49:03.754224 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:49:03.754224 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:49:03.754224 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 19:49:03.754224 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:03.754224 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 401/795] 10.222.0.44 () {56 vars in 1093 bytes} [Thu Jun 23 19:49:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:03.760437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d83e69-5d36-47ff-9a6f-0587dc06885e None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:03.760437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d83e69-5d36-47ff-9a6f-0587dc06885e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:03.760437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d83e69-5d36-47ff-9a6f-0587dc06885e None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:03.760437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 402/796] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:03.761939 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-2b5cb3c4-6264-4db7-a806-8aee1ff947fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:03.761939 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:03.761939 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:03.761939 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:03.761939 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:03.761939 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:03.761939 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:03.761939 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:03.761939 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:03.761939 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:03.761939 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:03.761939 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:03.761939 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:49:03.761939 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:49:03.761939 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:49:03.763801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:49:03.763801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:03.763801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:03.763801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:03.763801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:03.763801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:03.763801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:03.763801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:03.763801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:03.763801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:03.763801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:03.763801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:03.763801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:03.763801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:03.763801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:03.763801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:03.763801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:03.764950 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:49:03.764950 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:49:03.764950 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:03.764950 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:03.765233 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d83e69-5d36-47ff-9a6f-0587dc06885e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:03.765318 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 395/797] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:03.771387 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5cb3c4-6264-4db7-a806-8aee1ff947fe None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:03.816090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:03.872882 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d83e69-5d36-47ff-9a6f-0587dc06885e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:03.881975 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5cb3c4-6264-4db7-a806-8aee1ff947fe None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:03.908565 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308b903-b192-4505-8c66-1830186b47aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:03.909239 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308b903-b192-4505-8c66-1830186b47aa None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:03.909518 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308b903-b192-4505-8c66-1830186b47aa None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:03.909790 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308b903-b192-4505-8c66-1830186b47aa None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:03.911255 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308b903-b192-4505-8c66-1830186b47aa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:03.911643 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308b903-b192-4505-8c66-1830186b47aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:03.913301 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308b903-b192-4505-8c66-1830186b47aa None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:03.925436 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf524e-bec9-4a60-85c1-39f224ac34fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:03.926815 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf524e-bec9-4a60-85c1-39f224ac34fa None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:03.927289 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf524e-bec9-4a60-85c1-39f224ac34fa None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:03.927848 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf524e-bec9-4a60-85c1-39f224ac34fa None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:03.928950 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcf524e-bec9-4a60-85c1-39f224ac34fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:03.929966 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcf524e-bec9-4a60-85c1-39f224ac34fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:03.931510 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 403/798] 10.222.0.44 () {56 vars in 1133 bytes} [Thu Jun 23 19:49:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:03.932829 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a14832-da2c-4b8b-909e-f5ec1c710fa3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:03.933118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a14832-da2c-4b8b-909e-f5ec1c710fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:03.933397 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a14832-da2c-4b8b-909e-f5ec1c710fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:03.933653 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcf524e-bec9-4a60-85c1-39f224ac34fa None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:03.954885 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-81a14832-da2c-4b8b-909e-f5ec1c710fa3 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:03.954885 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-81a14832-da2c-4b8b-909e-f5ec1c710fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:03.955506 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 396/799] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:49:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:03.958004 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf524e-bec9-4a60-85c1-39f224ac34fa None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:03.966124 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:04.032401 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 404/800] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:04.032401 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a14832-da2c-4b8b-909e-f5ec1c710fa3 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:04.091763 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf524e-bec9-4a60-85c1-39f224ac34fa None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:04.101461 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:04.136614 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def51bd-bf77-4176-9bf6-2b08f979c5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:04.137639 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def51bd-bf77-4176-9bf6-2b08f979c5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:04.138619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def51bd-bf77-4176-9bf6-2b08f979c5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:04.139110 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def51bd-bf77-4176-9bf6-2b08f979c5b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:04.163368 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a14832-da2c-4b8b-909e-f5ec1c710fa3 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:04.199272 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33998c7-ec80-4d40-8859-c525281a1f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:04.200181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33998c7-ec80-4d40-8859-c525281a1f85 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:04.200489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33998c7-ec80-4d40-8859-c525281a1f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:04.200811 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33998c7-ec80-4d40-8859-c525281a1f85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:04.208651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33998c7-ec80-4d40-8859-c525281a1f85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:04.210117 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33998c7-ec80-4d40-8859-c525281a1f85 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:04.223390 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def51bd-bf77-4176-9bf6-2b08f979c5b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:04.225327 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def51bd-bf77-4176-9bf6-2b08f979c5b8 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:04.226228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-f33998c7-ec80-4d40-8859-c525281a1f85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:49:04.226228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:04.226228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:04.226228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:04.226228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:04.226228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:04.226228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:04.226228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:04.226228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:04.226228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:04.226228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:04.226228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:04.226228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:49:04.226228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:49:04.226228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:49:04.227611 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:49:04.227611 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:04.227611 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:04.227611 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:04.227611 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:04.227611 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:04.227611 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:04.227611 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:04.227611 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:04.227611 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:04.227611 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:04.227611 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:04.227611 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:04.227611 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:04.227611 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:04.227611 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:04.227611 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:04.228817 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:49:04.228817 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:49:04.228817 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:49:04.228817 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:49:04.228817 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:49:04.228817 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:49:04.228817 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:49:04.228817 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:04.232498 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 405/801] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:49:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:04.233545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33998c7-ec80-4d40-8859-c525281a1f85 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:04.244165 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:04.330943 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:04.340828 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 397/802] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:49:03 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 19:49:04.340828 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def51bd-bf77-4176-9bf6-2b08f979c5b8 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:04.346595 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33998c7-ec80-4d40-8859-c525281a1f85 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:04.382720 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9a93c-3ef0-49aa-9194-983fa7b353ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:04.383659 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9a93c-3ef0-49aa-9194-983fa7b353ef None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:04.384030 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9a93c-3ef0-49aa-9194-983fa7b353ef None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:04.384249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9a93c-3ef0-49aa-9194-983fa7b353ef None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:04.384934 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b9a93c-3ef0-49aa-9194-983fa7b353ef None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:04.386325 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b9a93c-3ef0-49aa-9194-983fa7b353ef None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:04.399041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-43b9a93c-3ef0-49aa-9194-983fa7b353ef None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 19:49:04.399041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:04.399041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 19:49:04.399041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 19:49:04.399041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 19:49:04.399041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 19:49:04.399041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 19:49:04.399041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:04.399041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 19:49:04.399041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:04.399041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:49:04.399041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:04.399041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:04.399041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:04.399041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:04.399041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:04.400009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:04.400009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:04.400009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:04.400009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:04.400009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:04.400009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:04.400009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:04.400009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 19:49:04.400009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 19:49:04.400009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 19:49:04.400009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 19:49:04.400009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:04.400009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:04.400009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:04.400009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:04.400009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:04.400878 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:04.400878 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:04.400878 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:04.400878 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:04.400878 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:04.400878 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:04.400878 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:04.400878 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:04.400878 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:04.400878 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:04.400878 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:04.400878 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 19:49:04.400878 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:04.400878 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 19:49:04.400878 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:04.405165 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 406/803] 10.222.0.44 () {56 vars in 1129 bytes} [Thu Jun 23 19:49:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:04.406460 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93d8f7-04cb-4393-b7a1-af7feeafdf3f None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:04.406747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93d8f7-04cb-4393-b7a1-af7feeafdf3f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:04.407026 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93d8f7-04cb-4393-b7a1-af7feeafdf3f None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:04.412407 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 407/804] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:04.412407 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93d8f7-04cb-4393-b7a1-af7feeafdf3f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:04.449256 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def51bd-bf77-4176-9bf6-2b08f979c5b8 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:04.482358 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:04.483363 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e221b3-f7be-4033-88ab-5484d3baf3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:04.484020 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e221b3-f7be-4033-88ab-5484d3baf3a3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:04.484325 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e221b3-f7be-4033-88ab-5484d3baf3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:04.484606 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e221b3-f7be-4033-88ab-5484d3baf3a3 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:04.485385 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e221b3-f7be-4033-88ab-5484d3baf3a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:04.485691 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e221b3-f7be-4033-88ab-5484d3baf3a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:04.487140 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e221b3-f7be-4033-88ab-5484d3baf3a3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:04.498995 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 398/805] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:49:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:04.499821 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e221b3-f7be-4033-88ab-5484d3baf3a3 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:04.523481 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93d8f7-04cb-4393-b7a1-af7feeafdf3f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:04.558450 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c106fe7-e095-435f-8ab8-b2b0b3432431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:04.559286 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c106fe7-e095-435f-8ab8-b2b0b3432431 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:04.559616 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c106fe7-e095-435f-8ab8-b2b0b3432431 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:04.559890 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c106fe7-e095-435f-8ab8-b2b0b3432431 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:04.563708 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:04.572078 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c106fe7-e095-435f-8ab8-b2b0b3432431 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:04.573559 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c106fe7-e095-435f-8ab8-b2b0b3432431 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:04.592048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-1c106fe7-e095-435f-8ab8-b2b0b3432431 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:04.592048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:04.592048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:04.592048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:04.592048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:04.592048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:04.592048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:04.592048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:04.592048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:04.592048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:04.592048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:04.592048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:04.592048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:49:04.592048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:49:04.592048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:49:04.593131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:49:04.593131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:04.593131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:04.593131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:04.593131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:04.593131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:04.593131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:04.593131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:04.593131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:04.593131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:04.593131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:04.593131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:04.593131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:04.593131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:04.593131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:04.593131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:04.593131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:04.594055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:49:04.594055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:49:04.594055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:04.594055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:04.597730 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 408/806] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:04.598929 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b485d-e154-454d-a6bb-2f10afcbdad8 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:04.599224 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b485d-e154-454d-a6bb-2f10afcbdad8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:04.599499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b485d-e154-454d-a6bb-2f10afcbdad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:04.608637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e221b3-f7be-4033-88ab-5484d3baf3a3 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:04.616619 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9d8b485d-e154-454d-a6bb-2f10afcbdad8 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:04.617787 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9d8b485d-e154-454d-a6bb-2f10afcbdad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:04.643986 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a48db-61a3-4896-946a-b6b8a0448efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:04.644686 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a48db-61a3-4896-946a-b6b8a0448efd None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:04.645027 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a48db-61a3-4896-946a-b6b8a0448efd None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:04.645361 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a48db-61a3-4896-946a-b6b8a0448efd None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:04.646117 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a48db-61a3-4896-946a-b6b8a0448efd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:04.646424 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a48db-61a3-4896-946a-b6b8a0448efd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:04.647807 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30a48db-61a3-4896-946a-b6b8a0448efd None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:04.657173 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 399/807] 10.222.0.44 () {56 vars in 1124 bytes} [Thu Jun 23 19:49:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:04.658019 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a48db-61a3-4896-946a-b6b8a0448efd None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:04.696455 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 409/808] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:04.697400 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b485d-e154-454d-a6bb-2f10afcbdad8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:04.749556 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:04.760034 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a48db-61a3-4896-946a-b6b8a0448efd None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:04.782304 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:04.794134 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52757665-cae3-40e0-b490-cf7c0a5e334b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:04.795017 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52757665-cae3-40e0-b490-cf7c0a5e334b None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:04.795301 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52757665-cae3-40e0-b490-cf7c0a5e334b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:04.795576 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52757665-cae3-40e0-b490-cf7c0a5e334b None admin] PATH_INFO: `/v3/projects/155180ba63c446ca889a6eb3f94cc31a/users/1d37f369749143ce813c693adc5f2ec7/roles/5f27c72acb19431fa3223c962bc42938` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:04.795666 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b485d-e154-454d-a6bb-2f10afcbdad8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:04.809835 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52757665-cae3-40e0-b490-cf7c0a5e334b None admin] RBAC: Authorizing `identity:create_grant(project_id=155180ba63c446ca889a6eb3f94cc31a, user_id=1d37f369749143ce813c693adc5f2ec7, role_id=5f27c72acb19431fa3223c962bc42938)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:04.811275 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52757665-cae3-40e0-b490-cf7c0a5e334b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:04.829585 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15fbab-c4b0-41db-aa66-6e63c42cbd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:04.830242 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15fbab-c4b0-41db-aa66-6e63c42cbd00 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:04.830528 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15fbab-c4b0-41db-aa66-6e63c42cbd00 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:04.830805 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15fbab-c4b0-41db-aa66-6e63c42cbd00 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:04.831515 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15fbab-c4b0-41db-aa66-6e63c42cbd00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:04.831823 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15fbab-c4b0-41db-aa66-6e63c42cbd00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:04.833759 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15fbab-c4b0-41db-aa66-6e63c42cbd00 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:04.844666 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 410/809] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:49:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:04.845394 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15fbab-c4b0-41db-aa66-6e63c42cbd00 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:04.880307 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 400/810] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jun 23 19:49:04 2022] PUT /identity/v3/projects/155180ba63c446ca889a6eb3f94cc31a/users/1d37f369749143ce813c693adc5f2ec7/roles/5f27c72acb19431fa3223c962bc42938 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:49:04.881633 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd5ff2-9ae0-41c1-8489-a18e85fe2fe5 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:04.881920 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd5ff2-9ae0-41c1-8489-a18e85fe2fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:04.882194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd5ff2-9ae0-41c1-8489-a18e85fe2fe5 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:04.884241 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 401/811] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:04.884919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd5ff2-9ae0-41c1-8489-a18e85fe2fe5 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:04.963480 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15fbab-c4b0-41db-aa66-6e63c42cbd00 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:04.977202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:04.999427 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714b6fe-0d70-4895-b0c2-5671d72a9cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:05.000190 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714b6fe-0d70-4895-b0c2-5671d72a9cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:05.000483 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714b6fe-0d70-4895-b0c2-5671d72a9cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:05.000760 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714b6fe-0d70-4895-b0c2-5671d72a9cb9 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:05.001477 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714b6fe-0d70-4895-b0c2-5671d72a9cb9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:05.001804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714b6fe-0d70-4895-b0c2-5671d72a9cb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:05.003172 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714b6fe-0d70-4895-b0c2-5671d72a9cb9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:05.009307 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd5ff2-9ae0-41c1-8489-a18e85fe2fe5 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:05.009832 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 411/812] 10.222.0.44 () {56 vars in 1124 bytes} [Thu Jun 23 19:49:04 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:05.010292 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714b6fe-0d70-4895-b0c2-5671d72a9cb9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:05.056822 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab86b41-3643-45c8-87ce-eefb7a7c6b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:05.057620 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab86b41-3643-45c8-87ce-eefb7a7c6b97 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:05.057908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab86b41-3643-45c8-87ce-eefb7a7c6b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:05.058182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab86b41-3643-45c8-87ce-eefb7a7c6b97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:05.070563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab86b41-3643-45c8-87ce-eefb7a7c6b97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:49:05.078116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-4ab86b41-3643-45c8-87ce-eefb7a7c6b97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:49:05.078116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:05.078116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:05.078116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:05.078116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:05.078116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:05.078116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:05.078116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:05.078116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:05.078116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:05.078116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:05.078116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:05.078116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:49:05.078116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:49:05.078116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:49:05.078116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:49:05.079114 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:49:05.079114 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:49:05.079114 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:49:05.079114 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:49:05.079114 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:05.079114 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:05.079114 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:05.079114 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:05.079114 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:05.079114 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:05.079114 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:05.079114 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:05.079114 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:05.079114 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:05.079114 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:05.079114 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:05.080322 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:05.080322 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:05.080322 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:05.080322 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:05.080322 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:49:05.080322 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:49:05.080322 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:49:05.080322 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:49:05.080322 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:49:05.080322 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:49:05.080322 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:49:05.080322 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:05.080322 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 402/813] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:49:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:05.081186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835f272-0d12-4206-92c5-7495c5e07d55 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:05.081186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835f272-0d12-4206-92c5-7495c5e07d55 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:05.081186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835f272-0d12-4206-92c5-7495c5e07d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:05.096990 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-5835f272-0d12-4206-92c5-7495c5e07d55 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:05.098087 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5835f272-0d12-4206-92c5-7495c5e07d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:05.123150 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:05.133040 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714b6fe-0d70-4895-b0c2-5671d72a9cb9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:05.162516 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 403/814] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:05.163552 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835f272-0d12-4206-92c5-7495c5e07d55 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:05.173413 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7a1131-bcd3-4f2b-a1be-975808ad70c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:05.174749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a1131-bcd3-4f2b-a1be-975808ad70c2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:05.175201 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a1131-bcd3-4f2b-a1be-975808ad70c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:05.175875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a1131-bcd3-4f2b-a1be-975808ad70c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:05.187886 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7a1131-bcd3-4f2b-a1be-975808ad70c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:05.189998 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7a1131-bcd3-4f2b-a1be-975808ad70c2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:05.205504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-1f7a1131-bcd3-4f2b-a1be-975808ad70c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:49:05.205504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:05.205504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:05.205504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:05.205504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:05.205504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:05.205504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:05.205504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:05.205504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:05.205504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:05.205504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:05.205504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:05.205504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:49:05.205504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:49:05.205504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:49:05.206653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:49:05.206653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:05.206653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:05.206653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:05.206653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:05.206653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:05.206653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:05.206653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:05.206653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:05.206653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:05.206653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:05.206653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:05.206653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:05.206653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:05.206653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:05.206653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:05.206653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:05.207787 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:49:05.207787 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:49:05.207787 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:49:05.207787 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:49:05.207787 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:49:05.207787 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:49:05.207787 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:49:05.207787 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:05.212686 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 412/815] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:49:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:05.214472 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7a1131-bcd3-4f2b-a1be-975808ad70c2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:05.260017 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:05.271790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835f272-0d12-4206-92c5-7495c5e07d55 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:05.307318 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9219bf56-2968-4b49-8d6b-d4efb943efe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:05.307974 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219bf56-2968-4b49-8d6b-d4efb943efe2 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:05.308256 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219bf56-2968-4b49-8d6b-d4efb943efe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:05.308625 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219bf56-2968-4b49-8d6b-d4efb943efe2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:05.309285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9219bf56-2968-4b49-8d6b-d4efb943efe2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:05.310672 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9219bf56-2968-4b49-8d6b-d4efb943efe2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:05.329369 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7a1131-bcd3-4f2b-a1be-975808ad70c2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:05.363198 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ec1e0-8c6c-4a3b-b9e1-44be9370339a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:05.364369 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ec1e0-8c6c-4a3b-b9e1-44be9370339a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:05.364654 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ec1e0-8c6c-4a3b-b9e1-44be9370339a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:05.364928 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ec1e0-8c6c-4a3b-b9e1-44be9370339a None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:05.365660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ec1e0-8c6c-4a3b-b9e1-44be9370339a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:05.365962 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ec1e0-8c6c-4a3b-b9e1-44be9370339a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:05.367459 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ec1e0-8c6c-4a3b-b9e1-44be9370339a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:05.375523 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 413/816] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:49:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:05.376292 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ec1e0-8c6c-4a3b-b9e1-44be9370339a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:05.430932 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 404/817] 10.222.0.44 () {60 vars in 1144 bytes} [Thu Jun 23 19:49:05 2022] POST /identity/v3/endpoints => generated 346 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:49:05.431829 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9219bf56-2968-4b49-8d6b-d4efb943efe2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:05.461578 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:05.493464 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:05.500017 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ec1e0-8c6c-4a3b-b9e1-44be9370339a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:05.547824 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329991f3-ecb3-4c8b-8e15-b8ebbcaeddfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:05.549128 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329991f3-ecb3-4c8b-8e15-b8ebbcaeddfe None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:05.549643 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329991f3-ecb3-4c8b-8e15-b8ebbcaeddfe None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:05.550084 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329991f3-ecb3-4c8b-8e15-b8ebbcaeddfe None admin] PATH_INFO: `/v3/services/network` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:05.550921 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329991f3-ecb3-4c8b-8e15-b8ebbcaeddfe None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:05.553146 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329991f3-ecb3-4c8b-8e15-b8ebbcaeddfe None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:05.566660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-329991f3-ecb3-4c8b-8e15-b8ebbcaeddfe None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 19:49:05.566660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:05.566660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 19:49:05.566660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 19:49:05.566660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 19:49:05.566660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 19:49:05.566660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 19:49:05.566660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:05.566660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 19:49:05.566660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:05.566660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:49:05.566660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:05.566660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:05.566660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:05.566660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:05.566660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:05.566660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:05.567749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:05.567749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:05.567749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:05.567749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:05.567749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:05.567749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:05.567749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 19:49:05.567749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 19:49:05.567749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 19:49:05.567749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 19:49:05.567749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:05.567749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:05.567749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:05.567749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:05.567749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:05.567749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:05.568646 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:05.568646 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:05.568646 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:05.568646 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:05.568646 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:05.568646 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:05.568646 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:05.568646 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:05.568646 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:05.568646 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:05.568646 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 19:49:05.568646 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:05.568646 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 19:49:05.568646 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:05.576655 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 414/818] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:49:05 2022] GET /identity/v3/services/network => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:05.577367 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9219bf56-2968-4b49-8d6b-d4efb943efe2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:05.578205 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329991f3-ecb3-4c8b-8e15-b8ebbcaeddfe None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:05.628980 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb9cfd-8db4-4a8d-8c92-732d2cf3b3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:05.629609 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb9cfd-8db4-4a8d-8c92-732d2cf3b3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:05.630251 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb9cfd-8db4-4a8d-8c92-732d2cf3b3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:05.630532 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb9cfd-8db4-4a8d-8c92-732d2cf3b3e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:05.631387 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb9cfd-8db4-4a8d-8c92-732d2cf3b3e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:05.631689 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb9cfd-8db4-4a8d-8c92-732d2cf3b3e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:05.633921 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb9cfd-8db4-4a8d-8c92-732d2cf3b3e6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:05.648174 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 405/819] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:49:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:05.649056 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb9cfd-8db4-4a8d-8c92-732d2cf3b3e6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:05.699929 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329991f3-ecb3-4c8b-8e15-b8ebbcaeddfe None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:05.738189 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb4f5e-90f4-49d3-8e2e-fa0064b98093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:05.739528 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb4f5e-90f4-49d3-8e2e-fa0064b98093 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:05.740027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb4f5e-90f4-49d3-8e2e-fa0064b98093 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:05.740688 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb4f5e-90f4-49d3-8e2e-fa0064b98093 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:05.745243 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:05.753237 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cb4f5e-90f4-49d3-8e2e-fa0064b98093 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:05.766055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cb4f5e-90f4-49d3-8e2e-fa0064b98093 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:05.771751 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb9cfd-8db4-4a8d-8c92-732d2cf3b3e6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:05.772969 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-17cb4f5e-90f4-49d3-8e2e-fa0064b98093 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:05.772969 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:05.772969 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:05.772969 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:05.772969 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:05.772969 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:05.772969 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:05.772969 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:05.772969 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:05.772969 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:05.772969 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:05.772969 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:05.772969 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:49:05.772969 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:49:05.772969 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:49:05.773924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:49:05.773924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:05.773924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:05.773924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:05.773924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:05.773924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:05.773924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:05.773924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:05.773924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:05.773924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:05.773924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:05.773924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:05.773924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:05.773924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:05.773924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:05.773924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:05.773924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:05.775209 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:49:05.775209 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:49:05.775209 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:05.775209 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:05.775209 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 415/820] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:05.775495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb4f5e-90f4-49d3-8e2e-fa0064b98093 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:05.794292 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:05.817620 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d340d5-4017-45db-8fe0-7aaba5e0e9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:05.818552 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d340d5-4017-45db-8fe0-7aaba5e0e9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:05.818909 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d340d5-4017-45db-8fe0-7aaba5e0e9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:05.819259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d340d5-4017-45db-8fe0-7aaba5e0e9a9 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:05.820880 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d340d5-4017-45db-8fe0-7aaba5e0e9a9 None admin] RBAC: Adding query filter params (name=network) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:05.821267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d340d5-4017-45db-8fe0-7aaba5e0e9a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:05.822732 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d340d5-4017-45db-8fe0-7aaba5e0e9a9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:05.834015 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 406/821] 10.222.0.44 () {56 vars in 1112 bytes} [Thu Jun 23 19:49:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:05.834015 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d340d5-4017-45db-8fe0-7aaba5e0e9a9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:05.912876 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb4f5e-90f4-49d3-8e2e-fa0064b98093 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:05.951035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d340d5-4017-45db-8fe0-7aaba5e0e9a9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:05.955457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf42e3-b005-4750-b0d0-1a6790601507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:05.956117 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf42e3-b005-4750-b0d0-1a6790601507 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:05.956420 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf42e3-b005-4750-b0d0-1a6790601507 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:05.956693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf42e3-b005-4750-b0d0-1a6790601507 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:05.958095 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf42e3-b005-4750-b0d0-1a6790601507 None admin] RBAC: Adding query filter params (role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb, scope.project.id=155180ba63c446ca889a6eb3f94cc31a, user.id=a63d1d902c024c97a38f7d92f73814d3) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:05.958403 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf42e3-b005-4750-b0d0-1a6790601507 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:05.959939 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf42e3-b005-4750-b0d0-1a6790601507 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:05.966947 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 416/822] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:49:05 2022] GET /identity/v3/role_assignments?user.id=a63d1d902c024c97a38f7d92f73814d3&scope.project.id=155180ba63c446ca889a6eb3f94cc31a&role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:05.967871 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf42e3-b005-4750-b0d0-1a6790601507 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:06.001092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f83ada-3b9c-46d9-9b25-82f2ded92731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:06.001791 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f83ada-3b9c-46d9-9b25-82f2ded92731 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:06.002083 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f83ada-3b9c-46d9-9b25-82f2ded92731 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:06.002351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f83ada-3b9c-46d9-9b25-82f2ded92731 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:06.003126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f83ada-3b9c-46d9-9b25-82f2ded92731 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:06.003435 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f83ada-3b9c-46d9-9b25-82f2ded92731 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:06.010384 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f83ada-3b9c-46d9-9b25-82f2ded92731 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:06.040084 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 407/823] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:06.045607 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f83ada-3b9c-46d9-9b25-82f2ded92731 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:06.051111 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:06.144674 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:06.208508 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf42e3-b005-4750-b0d0-1a6790601507 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:06.226278 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f83ada-3b9c-46d9-9b25-82f2ded92731 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:06.263721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb855e3-3fa9-46a2-9df3-08f090cc34b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:06.265651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb855e3-3fa9-46a2-9df3-08f090cc34b9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:06.266347 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb855e3-3fa9-46a2-9df3-08f090cc34b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:06.267028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb855e3-3fa9-46a2-9df3-08f090cc34b9 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:06.268474 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb855e3-3fa9-46a2-9df3-08f090cc34b9 None admin] RBAC: Adding query filter params (type=network) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:06.269221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb855e3-3fa9-46a2-9df3-08f090cc34b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:06.276371 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb855e3-3fa9-46a2-9df3-08f090cc34b9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:06.276681 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdbcc89-5954-4a2b-a178-df05278403ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:06.278391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdbcc89-5954-4a2b-a178-df05278403ae None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:06.278391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdbcc89-5954-4a2b-a178-df05278403ae None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:06.278391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdbcc89-5954-4a2b-a178-df05278403ae None admin] PATH_INFO: `/v3/users/swift` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:06.286784 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 417/824] 10.222.0.44 () {56 vars in 1112 bytes} [Thu Jun 23 19:49:05 2022] GET /identity/v3/services?type=network => generated 350 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:06.292139 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdbcc89-5954-4a2b-a178-df05278403ae None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:06.293613 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb855e3-3fa9-46a2-9df3-08f090cc34b9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:06.293751 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdbcc89-5954-4a2b-a178-df05278403ae None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:06.324901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-dcdbcc89-5954-4a2b-a178-df05278403ae None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 19:49:06.324901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:06.324901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:06.324901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:06.324901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:06.324901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:06.324901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:06.324901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:06.324901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:06.324901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:06.324901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:06.324901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:06.324901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:49:06.324901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:49:06.324901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:49:06.324901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:49:06.325922 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:06.325922 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:06.325922 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:49:06.325922 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:49:06.325922 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:49:06.325922 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:49:06.325922 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:06.325922 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:06.325922 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:06.325922 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:06.325922 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:06.325922 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:06.325922 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:06.325922 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:06.325922 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:06.325922 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:06.325922 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:06.325922 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:06.326878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:06.326878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:06.326878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:49:06.326878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:49:06.326878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:49:06.326878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:49:06.326878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:49:06.326878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:49:06.326878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 19:49:06.326878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:06.327406 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 408/825] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:49:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:49:06.335037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdbcc89-5954-4a2b-a178-df05278403ae None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:06.459513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:06.482689 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb855e3-3fa9-46a2-9df3-08f090cc34b9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:06.533133 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2b42d-2efa-4842-a37b-2adb6cd2cf77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:06.535293 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2b42d-2efa-4842-a37b-2adb6cd2cf77 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:06.536769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2b42d-2efa-4842-a37b-2adb6cd2cf77 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:06.537256 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2b42d-2efa-4842-a37b-2adb6cd2cf77 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:06.538857 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a2b42d-2efa-4842-a37b-2adb6cd2cf77 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:06.541158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdbcc89-5954-4a2b-a178-df05278403ae None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:06.542990 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a2b42d-2efa-4842-a37b-2adb6cd2cf77 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:06.560290 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:06.592933 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f3fe91-f20c-4b41-a037-c8c3e48b7cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:06.594042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3fe91-f20c-4b41-a037-c8c3e48b7cda None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:06.594548 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3fe91-f20c-4b41-a037-c8c3e48b7cda None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:06.595028 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3fe91-f20c-4b41-a037-c8c3e48b7cda None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:06.596170 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f3fe91-f20c-4b41-a037-c8c3e48b7cda None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:06.596748 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f3fe91-f20c-4b41-a037-c8c3e48b7cda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:06.598593 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f3fe91-f20c-4b41-a037-c8c3e48b7cda None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:06.619485 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 409/826] 10.222.0.44 () {56 vars in 1135 bytes} [Thu Jun 23 19:49:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:06.625713 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f3fe91-f20c-4b41-a037-c8c3e48b7cda None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:06.662388 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 418/827] 10.222.0.44 () {60 vars in 1144 bytes} [Thu Jun 23 19:49:06 2022] POST /identity/v3/endpoints => generated 327 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 19:49:06.763415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f3fe91-f20c-4b41-a037-c8c3e48b7cda None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:06.903167 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:06.924968 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af64ee-4f79-462d-ae87-6a50911ff565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:06.925780 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af64ee-4f79-462d-ae87-6a50911ff565 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:06.926088 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af64ee-4f79-462d-ae87-6a50911ff565 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:06.926909 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af64ee-4f79-462d-ae87-6a50911ff565 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:06.938712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af64ee-4f79-462d-ae87-6a50911ff565 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:06.940194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af64ee-4f79-462d-ae87-6a50911ff565 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:06.959919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-92af64ee-4f79-462d-ae87-6a50911ff565 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:06.959919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:06.959919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:06.959919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:06.959919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:06.959919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:06.959919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:06.959919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:06.959919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:06.959919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:06.959919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:06.959919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:06.959919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:49:06.959919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:49:06.959919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:49:06.960891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:49:06.960891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:06.960891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:06.960891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:06.960891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:06.960891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:06.960891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:06.960891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:06.960891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:06.960891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:06.960891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:06.960891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:06.960891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:06.960891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:06.960891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:06.960891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:06.960891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:06.961778 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:49:06.961778 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:49:06.961778 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:06.961778 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:06.961778 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 410/828] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:06.967093 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2b42d-2efa-4842-a37b-2adb6cd2cf77 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:07.081804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2b42d-2efa-4842-a37b-2adb6cd2cf77 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:07.114053 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f929de4-e47a-4e5f-b7a9-a34900a02508 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:07.114608 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f929de4-e47a-4e5f-b7a9-a34900a02508 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:07.114608 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f929de4-e47a-4e5f-b7a9-a34900a02508 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:07.115944 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 411/829] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:07.121674 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385de89d-99b4-482b-a4c1-21cac2945e0c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:07.121674 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385de89d-99b4-482b-a4c1-21cac2945e0c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:07.121674 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385de89d-99b4-482b-a4c1-21cac2945e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:07.125088 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e975b9a-6da5-4279-b011-0be62044ca38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:07.125088 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e975b9a-6da5-4279-b011-0be62044ca38 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:07.125088 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e975b9a-6da5-4279-b011-0be62044ca38 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:07.125088 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e975b9a-6da5-4279-b011-0be62044ca38 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:07.125486 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e975b9a-6da5-4279-b011-0be62044ca38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:07.125486 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e975b9a-6da5-4279-b011-0be62044ca38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:07.126492 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e975b9a-6da5-4279-b011-0be62044ca38 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:07.139221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 419/830] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:49:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:07.146410 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e975b9a-6da5-4279-b011-0be62044ca38 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:07.150054 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-385de89d-99b4-482b-a4c1-21cac2945e0c None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:07.151490 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-385de89d-99b4-482b-a4c1-21cac2945e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:07.204268 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:07.262080 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 412/831] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 19:49:07.273503 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e975b9a-6da5-4279-b011-0be62044ca38 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:07.285132 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385de89d-99b4-482b-a4c1-21cac2945e0c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:07.319096 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba5b8e-20b8-49a9-b8f3-585fe48ef1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:07.320678 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba5b8e-20b8-49a9-b8f3-585fe48ef1d1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:07.321174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba5b8e-20b8-49a9-b8f3-585fe48ef1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:07.321642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba5b8e-20b8-49a9-b8f3-585fe48ef1d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:07.332095 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba5b8e-20b8-49a9-b8f3-585fe48ef1d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:07.334557 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba5b8e-20b8-49a9-b8f3-585fe48ef1d1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:07.356093 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-93ba5b8e-20b8-49a9-b8f3-585fe48ef1d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:49:07.356093 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:07.356093 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:07.356093 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:07.356093 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:07.356093 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:07.356093 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:07.356093 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:07.356093 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:07.356093 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:07.356093 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:07.356093 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:07.356093 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:49:07.356093 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:49:07.356093 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:49:07.357182 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:49:07.357182 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:07.357182 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:07.357182 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:07.357182 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:07.357182 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:07.357182 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:07.357182 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:07.357182 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:07.357182 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:07.357182 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:07.357182 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:07.357182 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:07.357182 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:07.357182 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:07.357182 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:07.357182 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:07.358160 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:49:07.358160 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:49:07.358160 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:49:07.358160 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:49:07.358160 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:49:07.358160 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:49:07.358160 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:49:07.358160 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:07.358160 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 420/832] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:49:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:07.364379 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba5b8e-20b8-49a9-b8f3-585fe48ef1d1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:07.400388 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:07.404360 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385de89d-99b4-482b-a4c1-21cac2945e0c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:07.441433 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f302fc-7a07-4eac-b6f9-8134d6392197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:07.442821 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f302fc-7a07-4eac-b6f9-8134d6392197 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:07.443098 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f302fc-7a07-4eac-b6f9-8134d6392197 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:07.443364 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f302fc-7a07-4eac-b6f9-8134d6392197 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:07.455087 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f302fc-7a07-4eac-b6f9-8134d6392197 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:49:07.463619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-59f302fc-7a07-4eac-b6f9-8134d6392197 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:49:07.463619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:07.463619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:07.463619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:07.463619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:07.463619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:07.463619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:07.463619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:07.463619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:07.463619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:07.463619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:07.463619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:07.463619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:49:07.463619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:49:07.463619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:49:07.463619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:49:07.464838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:49:07.464838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:49:07.464838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:49:07.464838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:49:07.464838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:07.464838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:07.464838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:07.464838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:07.464838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:07.464838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:07.464838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:07.464838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:07.464838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:07.464838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:07.464838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:07.464838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:07.465728 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:07.465728 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:07.465728 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:07.465728 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:07.465728 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:49:07.465728 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:49:07.465728 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:49:07.465728 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:49:07.465728 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:49:07.465728 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:49:07.465728 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:49:07.465728 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:07.468127 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 413/833] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jun 23 19:49:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:07.475303 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f302fc-7a07-4eac-b6f9-8134d6392197 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:07.495133 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba5b8e-20b8-49a9-b8f3-585fe48ef1d1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:07.511035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:07.531684 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b147433-a408-42a4-9928-5ed2fcdee5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:07.532521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b147433-a408-42a4-9928-5ed2fcdee5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:07.532900 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b147433-a408-42a4-9928-5ed2fcdee5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:07.533269 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b147433-a408-42a4-9928-5ed2fcdee5e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:07.534139 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b147433-a408-42a4-9928-5ed2fcdee5e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:07.534542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b147433-a408-42a4-9928-5ed2fcdee5e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:07.536143 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b147433-a408-42a4-9928-5ed2fcdee5e8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:07.548625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 421/834] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:49:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:07.557778 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b147433-a408-42a4-9928-5ed2fcdee5e8 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:07.593649 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f302fc-7a07-4eac-b6f9-8134d6392197 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:07.630940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9072d-3a2c-419d-ad66-0e55ceedbebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:07.632094 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9072d-3a2c-419d-ad66-0e55ceedbebc None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:07.632549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9072d-3a2c-419d-ad66-0e55ceedbebc None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:07.632981 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9072d-3a2c-419d-ad66-0e55ceedbebc None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:07.634020 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9072d-3a2c-419d-ad66-0e55ceedbebc None admin] RBAC: Adding query filter params (name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:07.634540 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9072d-3a2c-419d-ad66-0e55ceedbebc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:07.636073 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9072d-3a2c-419d-ad66-0e55ceedbebc None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:07.650865 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 414/835] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jun 23 19:49:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:07.651823 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9072d-3a2c-419d-ad66-0e55ceedbebc None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:07.673915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b147433-a408-42a4-9928-5ed2fcdee5e8 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:07.674504 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:07.720505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aff7c1-5ab3-4acd-9c1b-e0b399ce6839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:07.721637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aff7c1-5ab3-4acd-9c1b-e0b399ce6839 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:07.721919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aff7c1-5ab3-4acd-9c1b-e0b399ce6839 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:07.722193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aff7c1-5ab3-4acd-9c1b-e0b399ce6839 None admin] PATH_INFO: `/v3/projects/155180ba63c446ca889a6eb3f94cc31a/users/96db03079816461bb322633f64b9fb9d/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:07.725438 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aff7c1-5ab3-4acd-9c1b-e0b399ce6839 None admin] RBAC: Authorizing `identity:create_grant(project_id=155180ba63c446ca889a6eb3f94cc31a, user_id=96db03079816461bb322633f64b9fb9d, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:07.727136 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aff7c1-5ab3-4acd-9c1b-e0b399ce6839 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:07.762787 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9072d-3a2c-419d-ad66-0e55ceedbebc None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:07.801915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 422/836] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jun 23 19:49:07 2022] PUT /identity/v3/projects/155180ba63c446ca889a6eb3f94cc31a/users/96db03079816461bb322633f64b9fb9d/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:49:07.805157 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549565f-2b81-4b65-b55e-51e09837e480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:07.806313 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549565f-2b81-4b65-b55e-51e09837e480 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:07.806746 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549565f-2b81-4b65-b55e-51e09837e480 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:07.807177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549565f-2b81-4b65-b55e-51e09837e480 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:07.820917 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549565f-2b81-4b65-b55e-51e09837e480 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:07.822865 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549565f-2b81-4b65-b55e-51e09837e480 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:07.844357 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-8549565f-2b81-4b65-b55e-51e09837e480 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:07.844357 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:07.844357 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:07.844357 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:07.844357 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:07.844357 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:07.844357 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:07.844357 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:07.844357 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:07.844357 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:07.844357 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:07.844357 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:07.844357 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:49:07.844357 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:49:07.844357 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:49:07.845510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:49:07.845510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:07.845510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:07.845510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:07.845510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:07.845510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:07.845510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:07.845510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:07.845510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:07.845510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:07.845510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:07.845510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:07.845510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:07.845510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:07.845510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:07.845510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:07.845510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:07.846603 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:49:07.846603 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:49:07.846603 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:07.846603 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:07.849455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 415/837] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:07.855419 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aff7c1-5ab3-4acd-9c1b-e0b399ce6839 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:07.862218 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f5c86-e1b5-4228-9580-cb64eeb8f9bd None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:07.862860 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f5c86-e1b5-4228-9580-cb64eeb8f9bd None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:07.863332 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f5c86-e1b5-4228-9580-cb64eeb8f9bd None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:07.865332 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 416/838] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:07.872456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf38eb7-690e-488f-b554-23e2f4664b6e None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:07.872990 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf38eb7-690e-488f-b554-23e2f4664b6e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:07.873435 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf38eb7-690e-488f-b554-23e2f4664b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:07.897868 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-5bf38eb7-690e-488f-b554-23e2f4664b6e None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:07.899347 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:07.899562 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5bf38eb7-690e-488f-b554-23e2f4664b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:07.990281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aff7c1-5ab3-4acd-9c1b-e0b399ce6839 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:07.990710 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 417/839] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:07 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:08.018649 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf38eb7-690e-488f-b554-23e2f4664b6e None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:08.021751 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:08.030913 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf5659-6fac-495f-b035-4cfe25afe149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:08.035550 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf5659-6fac-495f-b035-4cfe25afe149 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:08.035550 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf5659-6fac-495f-b035-4cfe25afe149 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:08.035550 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbf5659-6fac-495f-b035-4cfe25afe149 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:08.035550 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbf5659-6fac-495f-b035-4cfe25afe149 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:08.035550 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbf5659-6fac-495f-b035-4cfe25afe149 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:08.035550 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbf5659-6fac-495f-b035-4cfe25afe149 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:08.047342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 423/840] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:49:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:08.055847 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf5659-6fac-495f-b035-4cfe25afe149 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:08.138912 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf38eb7-690e-488f-b554-23e2f4664b6e None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:08.174076 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:08.181131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbf5659-6fac-495f-b035-4cfe25afe149 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:08.182902 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf4a24-0637-407a-845d-f9955af90306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:08.186992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf4a24-0637-407a-845d-f9955af90306 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:08.186992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf4a24-0637-407a-845d-f9955af90306 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:08.186992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf4a24-0637-407a-845d-f9955af90306 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:08.186992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf4a24-0637-407a-845d-f9955af90306 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:08.186992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf4a24-0637-407a-845d-f9955af90306 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:08.200813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-a4cf4a24-0637-407a-845d-f9955af90306 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 19:49:08.200813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:08.200813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 19:49:08.200813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 19:49:08.200813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 19:49:08.200813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 19:49:08.200813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 19:49:08.200813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:08.200813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 19:49:08.200813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:08.200813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:49:08.200813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:08.200813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:08.200813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:08.200813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:08.200813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:08.200813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:08.202066 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:08.202066 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:08.202066 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:08.202066 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:08.202066 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:08.202066 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:08.202066 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 19:49:08.202066 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 19:49:08.202066 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 19:49:08.202066 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 19:49:08.202066 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:08.202066 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:08.202066 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:08.202066 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:08.202066 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:08.202066 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:08.203193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:08.203193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:08.203193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:08.203193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:08.203193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:08.203193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:08.203193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:08.203193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:08.203193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:08.203193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:08.203193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 19:49:08.203193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:08.203193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 19:49:08.203193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:08.212277 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 418/841] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jun 23 19:49:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:08.213413 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf4a24-0637-407a-845d-f9955af90306 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:08.221043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e5909-e64c-4b66-982c-66ad09c69eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:08.221860 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e5909-e64c-4b66-982c-66ad09c69eaf None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:08.222140 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e5909-e64c-4b66-982c-66ad09c69eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:08.222413 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e5909-e64c-4b66-982c-66ad09c69eaf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:08.231555 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e5909-e64c-4b66-982c-66ad09c69eaf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:08.233286 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e5909-e64c-4b66-982c-66ad09c69eaf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:08.251552 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-242e5909-e64c-4b66-982c-66ad09c69eaf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 19:49:08.251552 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:08.251552 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:08.251552 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:08.251552 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:08.251552 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:08.251552 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:08.251552 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:08.251552 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:08.251552 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:08.251552 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:08.251552 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:08.251552 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:49:08.251552 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:49:08.251552 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:49:08.251552 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:49:08.252667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:08.252667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:08.252667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:49:08.252667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:49:08.252667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:49:08.252667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:49:08.252667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:08.252667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:08.252667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:08.252667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:08.252667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:08.252667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:08.252667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:08.252667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:08.252667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:08.252667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:08.252667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:08.252667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:08.253621 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:08.253621 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:08.253621 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:49:08.253621 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:49:08.253621 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:49:08.253621 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:49:08.253621 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:49:08.253621 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:49:08.253621 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 19:49:08.253621 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:08.256924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 424/842] 10.222.0.44 () {56 vars in 1117 bytes} [Thu Jun 23 19:49:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:49:08.265071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e5909-e64c-4b66-982c-66ad09c69eaf None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:08.293813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:08.333331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf4a24-0637-407a-845d-f9955af90306 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:08.369525 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244cfeda-565d-4d0f-8d41-b87fc8bac51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:08.370460 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244cfeda-565d-4d0f-8d41-b87fc8bac51f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:08.370951 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244cfeda-565d-4d0f-8d41-b87fc8bac51f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:08.371288 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244cfeda-565d-4d0f-8d41-b87fc8bac51f None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:08.372074 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244cfeda-565d-4d0f-8d41-b87fc8bac51f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:08.372483 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244cfeda-565d-4d0f-8d41-b87fc8bac51f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:08.373950 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244cfeda-565d-4d0f-8d41-b87fc8bac51f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:08.374320 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e5909-e64c-4b66-982c-66ad09c69eaf None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:08.385166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 419/843] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jun 23 19:49:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:08.385166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f5523-e912-4911-833e-53a0864dc2e9 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:08.385166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f5523-e912-4911-833e-53a0864dc2e9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:08.385166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005f5523-e912-4911-833e-53a0864dc2e9 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:08.385166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 420/844] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:08.388836 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f5523-e912-4911-833e-53a0864dc2e9 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:08.413728 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec087ec2-67a7-4038-897e-5b39840d21cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:08.414332 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec087ec2-67a7-4038-897e-5b39840d21cc None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:08.414613 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec087ec2-67a7-4038-897e-5b39840d21cc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:08.414887 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec087ec2-67a7-4038-897e-5b39840d21cc None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:08.415721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec087ec2-67a7-4038-897e-5b39840d21cc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:08.416025 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec087ec2-67a7-4038-897e-5b39840d21cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:08.417478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec087ec2-67a7-4038-897e-5b39840d21cc None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:08.434652 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 425/845] 10.222.0.44 () {56 vars in 1149 bytes} [Thu Jun 23 19:49:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:08.440399 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1d329-91f4-4d36-a019-d084cc8812eb None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:08.442244 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1d329-91f4-4d36-a019-d084cc8812eb None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:08.442244 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1d329-91f4-4d36-a019-d084cc8812eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:08.462542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c8c1d329-91f4-4d36-a019-d084cc8812eb None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:08.463735 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c8c1d329-91f4-4d36-a019-d084cc8812eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:08.471213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:08.498027 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005f5523-e912-4911-833e-53a0864dc2e9 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:08.528892 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:08.535225 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 426/846] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:08 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:08.535872 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef9d86-4eed-4719-a836-bdbdbcad5f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:08.536689 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef9d86-4eed-4719-a836-bdbdbcad5f7d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:08.541835 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef9d86-4eed-4719-a836-bdbdbcad5f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:08.541957 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1d329-91f4-4d36-a019-d084cc8812eb None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:08.542051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef9d86-4eed-4719-a836-bdbdbcad5f7d None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:08.542051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef9d86-4eed-4719-a836-bdbdbcad5f7d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:08.542051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef9d86-4eed-4719-a836-bdbdbcad5f7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:08.542051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef9d86-4eed-4719-a836-bdbdbcad5f7d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:08.544816 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 421/847] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jun 23 19:49:08 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:08.564242 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef9d86-4eed-4719-a836-bdbdbcad5f7d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:08.666621 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1d329-91f4-4d36-a019-d084cc8812eb None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:08.695669 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef9d86-4eed-4719-a836-bdbdbcad5f7d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:08.711223 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296323d-897c-4ec3-b36e-18cb11a80629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:08.712318 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296323d-897c-4ec3-b36e-18cb11a80629 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:08.712700 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296323d-897c-4ec3-b36e-18cb11a80629 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:08.713088 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296323d-897c-4ec3-b36e-18cb11a80629 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:08.728066 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:08.738219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b291c-c64d-411c-95e4-ed2af57ff2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:08.743351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b291c-c64d-411c-95e4-ed2af57ff2ed None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:08.743351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b291c-c64d-411c-95e4-ed2af57ff2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:08.743351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601b291c-c64d-411c-95e4-ed2af57ff2ed None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:08.743351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601b291c-c64d-411c-95e4-ed2af57ff2ed None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:08.743351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601b291c-c64d-411c-95e4-ed2af57ff2ed None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:08.757405 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-601b291c-c64d-411c-95e4-ed2af57ff2ed None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 19:49:08.757405 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:08.757405 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 19:49:08.757405 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 19:49:08.757405 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 19:49:08.757405 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 19:49:08.757405 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 19:49:08.757405 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:08.757405 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 19:49:08.757405 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:08.757405 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:49:08.757405 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:08.757405 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:08.757405 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:08.757405 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:08.757405 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:08.758374 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:08.758374 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:08.758374 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:08.758374 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:08.758374 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:08.758374 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:08.758374 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:08.758374 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 19:49:08.758374 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 19:49:08.758374 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 19:49:08.758374 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 19:49:08.758374 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:08.758374 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:08.758374 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:08.758374 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:08.758374 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:08.759426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:08.759426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:08.759426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:08.759426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:08.759426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:08.759426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:08.759426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:08.759426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:08.759426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:08.759426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:08.759426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:08.759426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 19:49:08.759426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:08.759426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 19:49:08.759426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:08.760429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 422/848] 10.222.0.44 () {56 vars in 1132 bytes} [Thu Jun 23 19:49:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:49:08.768245 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b291c-c64d-411c-95e4-ed2af57ff2ed None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:08.796174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296323d-897c-4ec3-b36e-18cb11a80629 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:08.798728 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296323d-897c-4ec3-b36e-18cb11a80629 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:08.830940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:08.876716 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601b291c-c64d-411c-95e4-ed2af57ff2ed None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:08.906136 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 427/849] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:49:08 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 19:49:08.912348 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296323d-897c-4ec3-b36e-18cb11a80629 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:08.916007 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b8150-66ab-48e7-91c1-42df7dab577c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:08.916888 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b8150-66ab-48e7-91c1-42df7dab577c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:08.917208 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b8150-66ab-48e7-91c1-42df7dab577c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:08.918382 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b8150-66ab-48e7-91c1-42df7dab577c None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:08.918382 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963b8150-66ab-48e7-91c1-42df7dab577c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:08.918824 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963b8150-66ab-48e7-91c1-42df7dab577c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:08.920284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963b8150-66ab-48e7-91c1-42df7dab577c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:08.925748 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 423/850] 10.222.0.44 () {56 vars in 1126 bytes} [Thu Jun 23 19:49:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:08.931428 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b8150-66ab-48e7-91c1-42df7dab577c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:08.964201 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:09.042187 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296323d-897c-4ec3-b36e-18cb11a80629 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:09.070941 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963b8150-66ab-48e7-91c1-42df7dab577c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:09.087014 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23590f-85c5-4669-bf26-beaedbf0947d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:09.088709 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23590f-85c5-4669-bf26-beaedbf0947d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:09.089221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23590f-85c5-4669-bf26-beaedbf0947d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:09.089940 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23590f-85c5-4669-bf26-beaedbf0947d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:09.098045 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:09.103075 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23590f-85c5-4669-bf26-beaedbf0947d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:09.109619 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23590f-85c5-4669-bf26-beaedbf0947d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:09.113444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a476c2-7f33-4ab3-84eb-c7cdd2f8bb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:09.114199 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a476c2-7f33-4ab3-84eb-c7cdd2f8bb6d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:09.114553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a476c2-7f33-4ab3-84eb-c7cdd2f8bb6d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:09.115403 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a476c2-7f33-4ab3-84eb-c7cdd2f8bb6d None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:09.116200 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a476c2-7f33-4ab3-84eb-c7cdd2f8bb6d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:09.116613 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a476c2-7f33-4ab3-84eb-c7cdd2f8bb6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:09.118661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a476c2-7f33-4ab3-84eb-c7cdd2f8bb6d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:09.128045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-6d23590f-85c5-4669-bf26-beaedbf0947d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:09.128045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:09.128045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:09.128045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:09.128045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:09.128045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:09.128045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:09.128045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:09.128045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:09.128045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:09.128045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:09.128045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:09.128045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:49:09.128045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:49:09.128045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:49:09.129631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:49:09.129631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:09.129631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:09.129631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:09.129631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:09.129631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:09.129631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:09.129631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:09.129631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:09.129631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:09.129631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:09.129631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:09.129631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:09.129631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:09.129631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:09.129631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:09.129631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:09.130566 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:49:09.130566 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:49:09.130566 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:09.130566 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:09.131429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 424/851] 10.222.0.44 () {56 vars in 1126 bytes} [Thu Jun 23 19:49:08 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:09.133311 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 428/852] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:49:09.142161 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a476c2-7f33-4ab3-84eb-c7cdd2f8bb6d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:09.256384 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a476c2-7f33-4ab3-84eb-c7cdd2f8bb6d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:09.297141 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dbc612-7f27-403b-ad0f-ef5ba0946cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:09.297813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbc612-7f27-403b-ad0f-ef5ba0946cae None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:09.298098 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbc612-7f27-403b-ad0f-ef5ba0946cae None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:09.298375 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbc612-7f27-403b-ad0f-ef5ba0946cae None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:09.299200 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dbc612-7f27-403b-ad0f-ef5ba0946cae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:09.299510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dbc612-7f27-403b-ad0f-ef5ba0946cae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:09.300982 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dbc612-7f27-403b-ad0f-ef5ba0946cae None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:09.314218 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 425/853] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:49:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:09.320178 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23590f-85c5-4669-bf26-beaedbf0947d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:09.423996 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:09.432170 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23590f-85c5-4669-bf26-beaedbf0947d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:09.467671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef803f5-c700-4924-ab6e-7797f3023da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:09.468496 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef803f5-c700-4924-ab6e-7797f3023da1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:09.468781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef803f5-c700-4924-ab6e-7797f3023da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:09.469067 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef803f5-c700-4924-ab6e-7797f3023da1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:09.475948 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef803f5-c700-4924-ab6e-7797f3023da1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:09.477495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef803f5-c700-4924-ab6e-7797f3023da1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:09.492594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-8ef803f5-c700-4924-ab6e-7797f3023da1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:49:09.492594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:09.492594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:09.492594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:09.492594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:09.492594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:09.492594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:09.492594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:09.492594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:09.492594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:09.492594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:09.492594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:09.492594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:49:09.492594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:49:09.492594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:49:09.493505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:49:09.493505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:09.493505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:09.493505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:09.493505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:09.493505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:09.493505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:09.493505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:09.493505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:09.493505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:09.493505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:09.493505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:09.493505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:09.493505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:09.493505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:09.493505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:09.493505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:09.494421 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:49:09.494421 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:49:09.494421 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:49:09.494421 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:49:09.494421 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:49:09.494421 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:49:09.494421 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:49:09.494421 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:09.494421 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 429/854] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:49:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:09.501408 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dbc612-7f27-403b-ad0f-ef5ba0946cae None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:09.548448 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:09.603995 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dbc612-7f27-403b-ad0f-ef5ba0946cae None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:09.639867 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429d4f2c-63fa-460c-8205-f87abe71a8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:09.640915 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d4f2c-63fa-460c-8205-f87abe71a8c9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:09.641436 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d4f2c-63fa-460c-8205-f87abe71a8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:09.641926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d4f2c-63fa-460c-8205-f87abe71a8c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:09.642915 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429d4f2c-63fa-460c-8205-f87abe71a8c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:09.643438 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429d4f2c-63fa-460c-8205-f87abe71a8c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:09.645645 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429d4f2c-63fa-460c-8205-f87abe71a8c9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:09.658261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 426/855] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:49:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:09.664492 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef803f5-c700-4924-ab6e-7797f3023da1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:09.775452 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef803f5-c700-4924-ab6e-7797f3023da1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:09.810203 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef6aa91-695f-4fc7-b3b5-cfb225ee99cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:09.810865 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6aa91-695f-4fc7-b3b5-cfb225ee99cb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:09.811152 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6aa91-695f-4fc7-b3b5-cfb225ee99cb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:09.811430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef6aa91-695f-4fc7-b3b5-cfb225ee99cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:09.812272 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef6aa91-695f-4fc7-b3b5-cfb225ee99cb None admin] RBAC: Adding query filter params (role.id=5f27c72acb19431fa3223c962bc42938, scope.project.id=155180ba63c446ca889a6eb3f94cc31a, user.id=1d37f369749143ce813c693adc5f2ec7) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:09.812875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef6aa91-695f-4fc7-b3b5-cfb225ee99cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:09.814315 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef6aa91-695f-4fc7-b3b5-cfb225ee99cb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:09.821329 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 430/856] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:49:09 2022] GET /identity/v3/role_assignments?user.id=1d37f369749143ce813c693adc5f2ec7&scope.project.id=155180ba63c446ca889a6eb3f94cc31a&role.id=5f27c72acb19431fa3223c962bc42938 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:10.019402 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e193f05-d104-4a5e-abc4-801a54566e25 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:10.019719 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e193f05-d104-4a5e-abc4-801a54566e25 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:10.020004 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e193f05-d104-4a5e-abc4-801a54566e25 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:10.021670 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 427/857] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:10.027028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5415b25-8360-495a-aa31-2601e2785ab1 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:10.027263 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5415b25-8360-495a-aa31-2601e2785ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:10.027594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5415b25-8360-495a-aa31-2601e2785ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:10.045714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b5415b25-8360-495a-aa31-2601e2785ab1 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:10.047213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b5415b25-8360-495a-aa31-2601e2785ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:10.063070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:10.117905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 431/858] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:10 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 19:49:10.141176 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e193f05-d104-4a5e-abc4-801a54566e25 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:10.256887 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e193f05-d104-4a5e-abc4-801a54566e25 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:10.264092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:10.291326 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e113918-75b0-4da2-bfa6-4a7d3f708e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:10.292057 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e113918-75b0-4da2-bfa6-4a7d3f708e66 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:10.292360 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e113918-75b0-4da2-bfa6-4a7d3f708e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:10.292696 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e113918-75b0-4da2-bfa6-4a7d3f708e66 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:10.309176 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e113918-75b0-4da2-bfa6-4a7d3f708e66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:49:10.314703 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-9e113918-75b0-4da2-bfa6-4a7d3f708e66 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:49:10.314703 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:10.314703 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:10.314703 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:10.314703 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:10.314703 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:10.314703 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:10.314703 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:10.314703 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:10.314703 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:10.314703 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:10.314703 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:10.314703 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:49:10.314703 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:49:10.314703 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:49:10.314703 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:49:10.315645 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:49:10.315645 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:49:10.315645 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:49:10.315645 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:49:10.315645 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:10.315645 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:10.315645 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:10.315645 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:10.315645 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:10.315645 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:10.315645 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:10.315645 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:10.315645 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:10.315645 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:10.315645 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:10.315645 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:10.316505 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:10.316505 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:10.316505 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:10.316505 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:10.316505 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:49:10.316505 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:49:10.316505 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:49:10.316505 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:49:10.316505 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:49:10.316505 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:49:10.316505 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:49:10.316505 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:10.316505 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 428/859] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:49:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:10.322708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5415b25-8360-495a-aa31-2601e2785ab1 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:10.429121 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5415b25-8360-495a-aa31-2601e2785ab1 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:10.466871 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82befecd-7b8f-4c20-ae9f-27b7ec26d44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:10.467876 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82befecd-7b8f-4c20-ae9f-27b7ec26d44e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:10.468376 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82befecd-7b8f-4c20-ae9f-27b7ec26d44e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:10.468858 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82befecd-7b8f-4c20-ae9f-27b7ec26d44e None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:10.469805 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82befecd-7b8f-4c20-ae9f-27b7ec26d44e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:10.470303 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82befecd-7b8f-4c20-ae9f-27b7ec26d44e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:10.471908 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82befecd-7b8f-4c20-ae9f-27b7ec26d44e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:10.481485 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 432/860] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:49:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:10.492332 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e113918-75b0-4da2-bfa6-4a7d3f708e66 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:10.589194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e113918-75b0-4da2-bfa6-4a7d3f708e66 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:10.624064 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de35e74-e83a-49aa-b313-bebc49586a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:10.624875 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de35e74-e83a-49aa-b313-bebc49586a1b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:10.625156 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de35e74-e83a-49aa-b313-bebc49586a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:10.625429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de35e74-e83a-49aa-b313-bebc49586a1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:10.635856 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de35e74-e83a-49aa-b313-bebc49586a1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:10.637359 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de35e74-e83a-49aa-b313-bebc49586a1b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:10.656146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-2de35e74-e83a-49aa-b313-bebc49586a1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:10.656146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:10.656146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:10.656146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:10.656146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:10.656146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:10.656146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:10.656146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:10.656146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:10.656146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:10.656146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:10.656146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:10.656146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:49:10.656146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:49:10.656146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:49:10.657126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:49:10.657126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:10.657126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:10.657126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:10.657126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:10.657126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:10.657126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:10.657126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:10.657126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:10.657126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:10.657126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:10.657126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:10.657126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:10.657126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:10.657126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:10.657126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:10.657126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:10.658010 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:49:10.658010 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:49:10.658010 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:10.658010 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:10.658010 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 429/861] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:10.663542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82befecd-7b8f-4c20-ae9f-27b7ec26d44e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:10.684156 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:10.773868 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82befecd-7b8f-4c20-ae9f-27b7ec26d44e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:10.793896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbc742-6e1f-45ad-84af-55b588c17387 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:10.794184 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbc742-6e1f-45ad-84af-55b588c17387 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:10.794455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dbc742-6e1f-45ad-84af-55b588c17387 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:10.795927 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 430/862] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:49:10.801190 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae316a-7aa5-4da1-8705-327c6ee58efc None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:10.801761 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae316a-7aa5-4da1-8705-327c6ee58efc None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:10.802346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae316a-7aa5-4da1-8705-327c6ee58efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:10.809992 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f016c4bd-3a81-43df-8488-be28f592205c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:10.810709 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016c4bd-3a81-43df-8488-be28f592205c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:10.810993 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016c4bd-3a81-43df-8488-be28f592205c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:10.811264 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016c4bd-3a81-43df-8488-be28f592205c None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:10.811988 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f016c4bd-3a81-43df-8488-be28f592205c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:10.812290 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f016c4bd-3a81-43df-8488-be28f592205c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:10.813947 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f016c4bd-3a81-43df-8488-be28f592205c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:10.822502 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e2ae316a-7aa5-4da1-8705-327c6ee58efc None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:10.824138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e2ae316a-7aa5-4da1-8705-327c6ee58efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:10.831352 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 433/863] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:10.832448 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f016c4bd-3a81-43df-8488-be28f592205c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:10.848752 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:10.900617 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 431/864] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:10 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 19:49:10.917563 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:10.921920 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae316a-7aa5-4da1-8705-327c6ee58efc None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:10.942521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f016c4bd-3a81-43df-8488-be28f592205c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:10.977457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31283b-3e36-46e8-82e6-e49db26a22a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:10.978018 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31283b-3e36-46e8-82e6-e49db26a22a7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:10.978297 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31283b-3e36-46e8-82e6-e49db26a22a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:10.978570 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31283b-3e36-46e8-82e6-e49db26a22a7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:10.987078 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc31283b-3e36-46e8-82e6-e49db26a22a7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:10.989062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc31283b-3e36-46e8-82e6-e49db26a22a7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:11.003631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-fc31283b-3e36-46e8-82e6-e49db26a22a7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 19:49:11.003631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:11.003631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:11.003631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:11.003631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:11.003631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:11.003631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:11.003631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:11.003631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:11.003631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:11.003631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:11.003631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:11.003631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:49:11.003631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:49:11.003631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:49:11.003631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:49:11.004753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:11.004753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:11.004753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:49:11.004753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:49:11.004753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:49:11.004753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:49:11.004753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:11.004753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:11.004753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:11.004753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:11.004753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:11.004753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:11.004753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:11.004753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:11.004753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:11.004753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:11.004753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:11.004753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:11.005675 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:11.005675 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:11.005675 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:49:11.005675 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:49:11.005675 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:49:11.005675 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:49:11.005675 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:49:11.005675 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:49:11.005675 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 19:49:11.005675 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:11.009765 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 434/865] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:49:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:49:11.010845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31283b-3e36-46e8-82e6-e49db26a22a7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:11.041294 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae316a-7aa5-4da1-8705-327c6ee58efc None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:11.091763 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d963b800-d7e8-4b3c-957a-53a3cca0e6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:11.092822 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963b800-d7e8-4b3c-957a-53a3cca0e6e5 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:11.093325 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963b800-d7e8-4b3c-957a-53a3cca0e6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:11.094204 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963b800-d7e8-4b3c-957a-53a3cca0e6e5 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:11.095130 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d963b800-d7e8-4b3c-957a-53a3cca0e6e5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:11.097106 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d963b800-d7e8-4b3c-957a-53a3cca0e6e5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:11.120633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31283b-3e36-46e8-82e6-e49db26a22a7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:11.150582 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:11.156379 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101cb66-a21e-48e5-93ed-26c64d6c1abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:11.157046 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101cb66-a21e-48e5-93ed-26c64d6c1abd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:11.157328 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101cb66-a21e-48e5-93ed-26c64d6c1abd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:11.157600 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101cb66-a21e-48e5-93ed-26c64d6c1abd None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:11.158484 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101cb66-a21e-48e5-93ed-26c64d6c1abd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:11.158793 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101cb66-a21e-48e5-93ed-26c64d6c1abd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:11.160375 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101cb66-a21e-48e5-93ed-26c64d6c1abd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:11.178237 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 435/866] 10.222.0.44 () {56 vars in 1135 bytes} [Thu Jun 23 19:49:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:11.183980 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101cb66-a21e-48e5-93ed-26c64d6c1abd None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:11.185888 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 432/867] 10.222.0.44 () {60 vars in 1141 bytes} [Thu Jun 23 19:49:10 2022] POST /identity/v3/services => generated 251 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 19:49:11.272987 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed3013-3392-4322-bfce-dbd2c9aeaa86 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:11.273656 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed3013-3392-4322-bfce-dbd2c9aeaa86 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:11.273656 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed3013-3392-4322-bfce-dbd2c9aeaa86 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:11.279029 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 433/868] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:49:11.280827 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d89e79-e058-4502-849d-da5095a07e11 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:11.281121 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d89e79-e058-4502-849d-da5095a07e11 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:11.281399 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d89e79-e058-4502-849d-da5095a07e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:11.300691 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-23d89e79-e058-4502-849d-da5095a07e11 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:11.302482 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-23d89e79-e058-4502-849d-da5095a07e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:11.303608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101cb66-a21e-48e5-93ed-26c64d6c1abd None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:11.340681 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:11.358525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504f698-1b98-4d99-bec5-18d0662ba60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:11.358736 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504f698-1b98-4d99-bec5-18d0662ba60e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:11.358945 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504f698-1b98-4d99-bec5-18d0662ba60e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:11.359133 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504f698-1b98-4d99-bec5-18d0662ba60e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:11.376371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 434/869] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:11 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 19:49:11.399177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d89e79-e058-4502-849d-da5095a07e11 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:11.420891 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:11.442296 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e504f698-1b98-4d99-bec5-18d0662ba60e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:11.443759 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e504f698-1b98-4d99-bec5-18d0662ba60e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:11.532712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d89e79-e058-4502-849d-da5095a07e11 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:11.562920 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 436/870] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:49:11 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 19:49:11.563953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504f698-1b98-4d99-bec5-18d0662ba60e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:11.581300 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be79eb5-bbaf-4dec-a928-fc25d97c7923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:11.582918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be79eb5-bbaf-4dec-a928-fc25d97c7923 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:11.583656 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be79eb5-bbaf-4dec-a928-fc25d97c7923 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:11.584117 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be79eb5-bbaf-4dec-a928-fc25d97c7923 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:11.585052 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be79eb5-bbaf-4dec-a928-fc25d97c7923 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:11.587106 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be79eb5-bbaf-4dec-a928-fc25d97c7923 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:11.653917 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:11.669576 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504f698-1b98-4d99-bec5-18d0662ba60e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:11.705184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217491b2-0e86-4475-899c-9d59d2301294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:11.705967 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217491b2-0e86-4475-899c-9d59d2301294 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:11.706245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217491b2-0e86-4475-899c-9d59d2301294 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:11.706514 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217491b2-0e86-4475-899c-9d59d2301294 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:11.715760 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:11.719461 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217491b2-0e86-4475-899c-9d59d2301294 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:11.721042 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217491b2-0e86-4475-899c-9d59d2301294 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:11.724193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 435/871] 10.222.0.44 () {60 vars in 1141 bytes} [Thu Jun 23 19:49:11 2022] POST /identity/v3/services => generated 269 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:49:11.745504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-217491b2-0e86-4475-899c-9d59d2301294 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:11.745504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:11.745504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:11.745504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:11.745504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:11.745504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:11.745504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:11.745504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:11.745504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:11.745504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:11.745504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:11.745504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:11.745504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:49:11.745504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:49:11.745504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:49:11.746490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:49:11.746490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:11.746490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:11.746490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:11.746490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:11.746490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:11.746490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:11.746490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:11.746490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:11.746490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:11.746490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:11.746490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:11.746490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:11.746490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:11.746490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:11.746490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:11.746490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:11.747368 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:49:11.747368 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:49:11.747368 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:11.747368 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:11.747368 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 437/872] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:11.754170 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be79eb5-bbaf-4dec-a928-fc25d97c7923 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:11.881717 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be79eb5-bbaf-4dec-a928-fc25d97c7923 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:11.918495 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f99b00f-a0cd-433a-9dd4-c6f99073c6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:11.919087 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99b00f-a0cd-433a-9dd4-c6f99073c6f6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:11.919369 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99b00f-a0cd-433a-9dd4-c6f99073c6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:11.919642 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99b00f-a0cd-433a-9dd4-c6f99073c6f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:11.920407 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f99b00f-a0cd-433a-9dd4-c6f99073c6f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:11.920716 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f99b00f-a0cd-433a-9dd4-c6f99073c6f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:11.922103 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f99b00f-a0cd-433a-9dd4-c6f99073c6f6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:11.934876 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 436/873] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:49:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:11.940608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217491b2-0e86-4475-899c-9d59d2301294 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:12.015387 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a68bf2-6057-4741-9eb9-45543a3f332f None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:12.015993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a68bf2-6057-4741-9eb9-45543a3f332f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:12.016541 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a68bf2-6057-4741-9eb9-45543a3f332f None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:12.018385 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 437/874] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:49:12.025111 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f592f07-64b3-40cc-9b5c-b10d538e1450 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:12.025398 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f592f07-64b3-40cc-9b5c-b10d538e1450 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:12.025678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f592f07-64b3-40cc-9b5c-b10d538e1450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:12.046031 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3f592f07-64b3-40cc-9b5c-b10d538e1450 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:12.047382 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3f592f07-64b3-40cc-9b5c-b10d538e1450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:12.055034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217491b2-0e86-4475-899c-9d59d2301294 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:12.097643 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc376e9e-11c7-4149-b701-f1ad9eccc3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:12.098666 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc376e9e-11c7-4149-b701-f1ad9eccc3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:12.098952 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc376e9e-11c7-4149-b701-f1ad9eccc3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:12.099232 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc376e9e-11c7-4149-b701-f1ad9eccc3f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:12.102853 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:12.108561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc376e9e-11c7-4149-b701-f1ad9eccc3f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:12.110776 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc376e9e-11c7-4149-b701-f1ad9eccc3f4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:12.116566 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:12.130108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-bc376e9e-11c7-4149-b701-f1ad9eccc3f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:49:12.130108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:12.130108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:12.130108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:12.130108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:12.130108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:12.130108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:12.130108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:12.130108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:12.130108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:12.130108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:12.130108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:12.130108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:49:12.130108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:49:12.130108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:49:12.131006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:49:12.131006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:12.131006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:12.131006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:12.131006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:12.131006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:12.131006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:12.131006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:12.131006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:12.131006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:12.131006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:12.131006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:12.131006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:12.131006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:12.131006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:12.131006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:12.131006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:12.132679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:49:12.132679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:49:12.132679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:49:12.132679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:49:12.132679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:49:12.132679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:49:12.132679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:49:12.132679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:12.132679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 438/875] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:49:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:12.133367 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 438/876] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:12 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 19:49:12.141244 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc376e9e-11c7-4149-b701-f1ad9eccc3f4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:12.157311 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f592f07-64b3-40cc-9b5c-b10d538e1450 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:12.259619 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc376e9e-11c7-4149-b701-f1ad9eccc3f4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:12.264001 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f592f07-64b3-40cc-9b5c-b10d538e1450 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:12.295213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b67e4d-482c-46a2-a9a5-09ed54129468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:12.295875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b67e4d-482c-46a2-a9a5-09ed54129468 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:12.296152 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b67e4d-482c-46a2-a9a5-09ed54129468 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:12.296450 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b67e4d-482c-46a2-a9a5-09ed54129468 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:12.297336 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b67e4d-482c-46a2-a9a5-09ed54129468 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:12.297644 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b67e4d-482c-46a2-a9a5-09ed54129468 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:12.299376 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b67e4d-482c-46a2-a9a5-09ed54129468 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:12.300171 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11276c6d-da09-459c-9d58-366d8ac9c492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:12.301573 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11276c6d-da09-459c-9d58-366d8ac9c492 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:12.301855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11276c6d-da09-459c-9d58-366d8ac9c492 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:12.302125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11276c6d-da09-459c-9d58-366d8ac9c492 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:12.313653 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11276c6d-da09-459c-9d58-366d8ac9c492 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:49:12.318890 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 439/877] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:49:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:12.319774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b67e4d-482c-46a2-a9a5-09ed54129468 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:12.322138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-11276c6d-da09-459c-9d58-366d8ac9c492 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 19:49:12.322138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:12.322138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:12.322138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:12.322138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:12.322138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:12.322138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:12.322138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:12.322138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:12.322138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:12.322138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:12.322138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:12.322138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:49:12.322138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:49:12.322138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:49:12.322138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:49:12.324444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:49:12.324444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:49:12.324444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:49:12.324444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:49:12.324444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:12.324444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:12.324444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:12.324444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:12.324444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:12.324444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:12.324444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:12.324444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:12.324444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:12.324444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:12.324444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:12.324444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:12.326510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:12.326510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:12.326510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:12.326510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:12.326510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:49:12.326510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:49:12.326510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:49:12.326510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:49:12.326510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:49:12.326510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:49:12.326510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 19:49:12.326510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:12.336394 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 439/878] 10.222.0.44 () {56 vars in 1120 bytes} [Thu Jun 23 19:49:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:12.336394 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11276c6d-da09-459c-9d58-366d8ac9c492 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:12.377834 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:12.378747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:12.424026 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b67e4d-482c-46a2-a9a5-09ed54129468 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:12.439547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11276c6d-da09-459c-9d58-366d8ac9c492 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:12.461287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425831a-96c0-413d-9427-7747c099210f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:12.461928 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425831a-96c0-413d-9427-7747c099210f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:12.462264 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425831a-96c0-413d-9427-7747c099210f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:12.462538 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425831a-96c0-413d-9427-7747c099210f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:12.463367 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6425831a-96c0-413d-9427-7747c099210f None admin] RBAC: Adding query filter params (role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb, scope.project.id=155180ba63c446ca889a6eb3f94cc31a, user.id=96db03079816461bb322633f64b9fb9d) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:12.463673 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6425831a-96c0-413d-9427-7747c099210f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:12.465140 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6425831a-96c0-413d-9427-7747c099210f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:12.476501 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 440/879] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:49:12 2022] GET /identity/v3/role_assignments?user.id=96db03079816461bb322633f64b9fb9d&scope.project.id=155180ba63c446ca889a6eb3f94cc31a&role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:12.480024 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0e9eb-4bad-4e7b-bedb-624946f4b74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:12.480754 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0e9eb-4bad-4e7b-bedb-624946f4b74e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:12.481037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0e9eb-4bad-4e7b-bedb-624946f4b74e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:12.481312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee0e9eb-4bad-4e7b-bedb-624946f4b74e None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:12.482084 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee0e9eb-4bad-4e7b-bedb-624946f4b74e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:12.482411 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee0e9eb-4bad-4e7b-bedb-624946f4b74e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:12.483953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee0e9eb-4bad-4e7b-bedb-624946f4b74e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:12.494920 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 440/880] 10.222.0.44 () {56 vars in 1115 bytes} [Thu Jun 23 19:49:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:12.500952 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425831a-96c0-413d-9427-7747c099210f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:12.613584 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425831a-96c0-413d-9427-7747c099210f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:12.639179 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:12.654080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57c73c-ca08-4ba9-95ea-a327737f76d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:12.654924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57c73c-ca08-4ba9-95ea-a327737f76d4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:12.655176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57c73c-ca08-4ba9-95ea-a327737f76d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:12.655445 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57c73c-ca08-4ba9-95ea-a327737f76d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:12.667623 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57c73c-ca08-4ba9-95ea-a327737f76d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:12.669195 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57c73c-ca08-4ba9-95ea-a327737f76d4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:12.686642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-ed57c73c-ca08-4ba9-95ea-a327737f76d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:12.686642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:12.686642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:12.686642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:12.686642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:12.686642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:12.686642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:12.686642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:12.686642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:12.686642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:12.686642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:12.686642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:12.686642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:49:12.686642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:49:12.686642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:49:12.687843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:49:12.687843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:12.687843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:12.687843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:12.687843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:12.687843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:12.687843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:12.687843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:12.687843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:12.687843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:12.687843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:12.687843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:12.687843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:12.687843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:12.687843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:12.687843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:12.687843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:12.688760 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:49:12.688760 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:49:12.688760 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:12.688760 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:12.688760 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 441/881] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:12.696072 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0e9eb-4bad-4e7b-bedb-624946f4b74e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:12.809461 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee0e9eb-4bad-4e7b-bedb-624946f4b74e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:12.817199 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:12.844455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c53ef-7fe6-4565-b441-0fe3abc2556a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:12.845218 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c53ef-7fe6-4565-b441-0fe3abc2556a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:12.845499 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c53ef-7fe6-4565-b441-0fe3abc2556a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:12.845774 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c53ef-7fe6-4565-b441-0fe3abc2556a None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:12.846548 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c53ef-7fe6-4565-b441-0fe3abc2556a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:12.846856 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c53ef-7fe6-4565-b441-0fe3abc2556a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:12.849016 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c53ef-7fe6-4565-b441-0fe3abc2556a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:12.861564 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 441/882] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:49:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:12.867679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57c73c-ca08-4ba9-95ea-a327737f76d4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:12.974709 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57c73c-ca08-4ba9-95ea-a327737f76d4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:13.012435 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4afa34c-403a-4661-8335-9ca4afec0feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:13.013312 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afa34c-403a-4661-8335-9ca4afec0feb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:13.013596 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afa34c-403a-4661-8335-9ca4afec0feb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:13.013871 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afa34c-403a-4661-8335-9ca4afec0feb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:13.022743 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4afa34c-403a-4661-8335-9ca4afec0feb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:13.024220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4afa34c-403a-4661-8335-9ca4afec0feb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:13.041247 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-f4afa34c-403a-4661-8335-9ca4afec0feb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 19:49:13.041247 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:13.041247 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:13.041247 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:13.041247 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:13.041247 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:13.041247 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:13.041247 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:13.041247 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:13.041247 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:13.041247 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:13.041247 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:13.041247 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:49:13.041247 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:49:13.041247 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:49:13.041247 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:49:13.042249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:13.042249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:13.042249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:49:13.042249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:49:13.042249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:49:13.042249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:49:13.042249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:13.042249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:13.042249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:13.042249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:13.042249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:13.042249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:13.042249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:13.042249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:13.042249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:13.042249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:13.042249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:13.042249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:13.043171 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:13.043171 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:13.043171 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:49:13.043171 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:49:13.043171 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:49:13.043171 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:49:13.043171 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:49:13.043171 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:49:13.043171 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 19:49:13.043171 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:13.043171 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 442/883] 10.222.0.44 () {56 vars in 1117 bytes} [Thu Jun 23 19:49:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:13.048435 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c53ef-7fe6-4565-b441-0fe3abc2556a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:13.152069 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c53ef-7fe6-4565-b441-0fe3abc2556a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:13.186815 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b37e05-b104-4282-bc23-db21bbdb6c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:13.187720 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b37e05-b104-4282-bc23-db21bbdb6c42 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:13.188057 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b37e05-b104-4282-bc23-db21bbdb6c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:13.188362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b37e05-b104-4282-bc23-db21bbdb6c42 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:13.189198 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b37e05-b104-4282-bc23-db21bbdb6c42 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:13.189507 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b37e05-b104-4282-bc23-db21bbdb6c42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:13.191066 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b37e05-b104-4282-bc23-db21bbdb6c42 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:13.208957 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 442/884] 10.222.0.44 () {56 vars in 1149 bytes} [Thu Jun 23 19:49:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:13.214719 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4afa34c-403a-4661-8335-9ca4afec0feb None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:13.248742 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:13.319284 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4afa34c-403a-4661-8335-9ca4afec0feb None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:13.356460 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196e3ac-c18e-40d8-8b55-95f34c2a58f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:13.357110 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196e3ac-c18e-40d8-8b55-95f34c2a58f3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:13.357391 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196e3ac-c18e-40d8-8b55-95f34c2a58f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:13.357680 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196e3ac-c18e-40d8-8b55-95f34c2a58f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:13.389919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c5bec-e66a-419d-8c03-e3389bc7e0dc None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:13.390440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c5bec-e66a-419d-8c03-e3389bc7e0dc None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:13.390440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c5bec-e66a-419d-8c03-e3389bc7e0dc None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:13.391950 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 443/885] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:49:13.399043 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906df490-2925-4d27-a4aa-d5c2b841f611 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:13.399444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906df490-2925-4d27-a4aa-d5c2b841f611 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:13.399816 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906df490-2925-4d27-a4aa-d5c2b841f611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:13.421833 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-906df490-2925-4d27-a4aa-d5c2b841f611 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:13.423364 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-906df490-2925-4d27-a4aa-d5c2b841f611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:13.435191 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6196e3ac-c18e-40d8-8b55-95f34c2a58f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:13.437050 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6196e3ac-c18e-40d8-8b55-95f34c2a58f3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:13.446083 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:13.488804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:13.510814 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 444/886] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:13 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:13.533137 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906df490-2925-4d27-a4aa-d5c2b841f611 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:13.556332 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 443/887] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:49:13 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 19:49:13.560196 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196e3ac-c18e-40d8-8b55-95f34c2a58f3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:13.647329 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906df490-2925-4d27-a4aa-d5c2b841f611 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:13.669976 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196e3ac-c18e-40d8-8b55-95f34c2a58f3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:13.683417 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3dbee3-c06b-41c8-818d-d8fceccc2963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:13.684582 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3dbee3-c06b-41c8-818d-d8fceccc2963 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:13.685030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3dbee3-c06b-41c8-818d-d8fceccc2963 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:13.685449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3dbee3-c06b-41c8-818d-d8fceccc2963 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:13.686289 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3dbee3-c06b-41c8-818d-d8fceccc2963 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:13.688153 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3dbee3-c06b-41c8-818d-d8fceccc2963 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:13.702131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-8e3dbee3-c06b-41c8-818d-d8fceccc2963 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 19:49:13.702131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:13.702131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 19:49:13.702131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 19:49:13.702131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 19:49:13.702131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 19:49:13.702131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 19:49:13.702131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:13.702131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 19:49:13.702131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:13.702131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:49:13.702131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:13.702131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:13.702131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:13.702131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:13.702131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:13.702131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:13.703162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:13.703162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:13.703162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:13.703162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:13.703162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:13.703162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:13.703162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 19:49:13.703162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 19:49:13.703162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 19:49:13.703162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 19:49:13.703162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:13.703162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:13.703162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:13.703162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:13.703162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:13.703162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:13.703992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:13.703992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:13.703992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:13.703992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:13.703992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:13.703992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:13.703992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:13.703992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:13.703992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:13.703992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:13.703992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 19:49:13.703992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:13.703992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 19:49:13.703992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:13.705594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:13.707113 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 445/888] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jun 23 19:49:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:49:13.713461 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3dbee3-c06b-41c8-818d-d8fceccc2963 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:13.721496 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3a037-9290-4af1-8857-743048cdeb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:13.722502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3a037-9290-4af1-8857-743048cdeb72 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:13.722814 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3a037-9290-4af1-8857-743048cdeb72 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:13.723088 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3a037-9290-4af1-8857-743048cdeb72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:13.738734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd3a037-9290-4af1-8857-743048cdeb72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:13.739679 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:13.741433 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd3a037-9290-4af1-8857-743048cdeb72 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:13.765109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-1fd3a037-9290-4af1-8857-743048cdeb72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:13.765109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:13.765109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:13.765109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:13.765109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:13.765109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:13.765109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:13.765109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:13.765109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:13.765109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:13.765109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:13.765109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:13.765109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:49:13.765109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:49:13.765109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:49:13.766059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:49:13.766059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:13.766059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:13.766059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:13.766059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:13.766059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:13.766059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:13.766059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:13.766059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:13.766059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:13.766059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:13.766059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:13.766059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:13.766059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:13.766059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:13.766059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:13.766059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:13.767078 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:49:13.767078 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:49:13.767078 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:13.767078 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:13.772104 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 444/889] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:49:13.773062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3a037-9290-4af1-8857-743048cdeb72 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:13.848012 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3dbee3-c06b-41c8-818d-d8fceccc2963 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:13.890581 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac794cf0-cacc-486f-9b0b-86967c0bfc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:13.891887 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac794cf0-cacc-486f-9b0b-86967c0bfc80 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:13.892382 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac794cf0-cacc-486f-9b0b-86967c0bfc80 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:13.893059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac794cf0-cacc-486f-9b0b-86967c0bfc80 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:13.893913 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac794cf0-cacc-486f-9b0b-86967c0bfc80 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:13.894512 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac794cf0-cacc-486f-9b0b-86967c0bfc80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:13.896705 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac794cf0-cacc-486f-9b0b-86967c0bfc80 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:13.898310 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3a037-9290-4af1-8857-743048cdeb72 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:13.907936 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 446/890] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jun 23 19:49:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:13.909074 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac794cf0-cacc-486f-9b0b-86967c0bfc80 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:13.940142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e40207-c961-4869-b55e-9312b0575849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:13.940760 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e40207-c961-4869-b55e-9312b0575849 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:13.941043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e40207-c961-4869-b55e-9312b0575849 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:13.941323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e40207-c961-4869-b55e-9312b0575849 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:13.942595 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e40207-c961-4869-b55e-9312b0575849 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:13.942903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e40207-c961-4869-b55e-9312b0575849 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:13.944273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e40207-c961-4869-b55e-9312b0575849 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:13.955552 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 445/891] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:49:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:13.956786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac73c9-5117-43a7-bcb5-a784860b0cd8 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:13.957095 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac73c9-5117-43a7-bcb5-a784860b0cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:13.957372 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac73c9-5117-43a7-bcb5-a784860b0cd8 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:13.958825 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 446/892] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:13.961782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac73c9-5117-43a7-bcb5-a784860b0cd8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:14.008565 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:14.012683 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:14.028704 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac794cf0-cacc-486f-9b0b-86967c0bfc80 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:14.066654 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85a45e-fe7e-4c0b-8e29-4931cdbf1e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:14.067555 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85a45e-fe7e-4c0b-8e29-4931cdbf1e2c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:14.067835 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85a45e-fe7e-4c0b-8e29-4931cdbf1e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:14.068109 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85a45e-fe7e-4c0b-8e29-4931cdbf1e2c None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:14.068850 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed85a45e-fe7e-4c0b-8e29-4931cdbf1e2c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:14.069154 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed85a45e-fe7e-4c0b-8e29-4931cdbf1e2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:14.070235 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac73c9-5117-43a7-bcb5-a784860b0cd8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:14.070534 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed85a45e-fe7e-4c0b-8e29-4931cdbf1e2c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:14.075588 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 447/893] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jun 23 19:49:13 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:14.076891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc2b9b-231c-4726-b911-80efd4c81b8a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:14.077181 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc2b9b-231c-4726-b911-80efd4c81b8a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:14.077458 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc2b9b-231c-4726-b911-80efd4c81b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:14.093989 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-34bc2b9b-231c-4726-b911-80efd4c81b8a None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:14.095178 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-34bc2b9b-231c-4726-b911-80efd4c81b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:14.110051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e3038-8303-40dc-9606-8632c589d2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:14.110848 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e3038-8303-40dc-9606-8632c589d2cf None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:14.111130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e3038-8303-40dc-9606-8632c589d2cf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:14.111403 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e3038-8303-40dc-9606-8632c589d2cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:14.118876 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e3038-8303-40dc-9606-8632c589d2cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:14.120428 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e3038-8303-40dc-9606-8632c589d2cf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:14.134607 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-519e3038-8303-40dc-9606-8632c589d2cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:49:14.134607 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:14.134607 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:14.134607 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:14.134607 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:14.134607 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:14.134607 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:14.134607 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:14.134607 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:14.134607 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:14.134607 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:14.134607 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:14.134607 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:49:14.134607 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:49:14.134607 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:49:14.135682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:49:14.135682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:14.135682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:14.135682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:14.135682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:14.135682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:14.135682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:14.135682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:14.135682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:14.135682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:14.135682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:14.135682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:14.135682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:14.135682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:14.135682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:14.135682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:14.135682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:14.136818 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:49:14.136818 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:49:14.136818 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:49:14.136818 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:49:14.136818 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:49:14.136818 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:49:14.136818 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:49:14.136818 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:14.141451 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 447/894] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:49:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:14.142293 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e3038-8303-40dc-9606-8632c589d2cf None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:14.168253 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 448/895] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:14 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:14.169058 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc2b9b-231c-4726-b911-80efd4c81b8a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:14.254795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:14.271091 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e3038-8303-40dc-9606-8632c589d2cf None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:14.274308 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:14.280641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc2b9b-231c-4726-b911-80efd4c81b8a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:14.307099 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b455d3-c72f-4fd5-8d34-8396399f458b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:14.307786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b455d3-c72f-4fd5-8d34-8396399f458b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:14.308068 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b455d3-c72f-4fd5-8d34-8396399f458b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:14.308367 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b455d3-c72f-4fd5-8d34-8396399f458b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:14.309151 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b455d3-c72f-4fd5-8d34-8396399f458b None admin] RBAC: Adding query filter params (interface=public, service_id=38210e8fd3d54d3d845a27551edc8734, region_id=RegionOne) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:14.309459 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b455d3-c72f-4fd5-8d34-8396399f458b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:14.310829 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b455d3-c72f-4fd5-8d34-8396399f458b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:14.315299 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791c74e-980e-4ff7-916f-043dfbf1c474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:14.320701 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791c74e-980e-4ff7-916f-043dfbf1c474 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:14.320931 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 448/896] 10.222.0.44 () {56 vars in 1251 bytes} [Thu Jun 23 19:49:14 2022] GET /identity/v3/endpoints?service_id=38210e8fd3d54d3d845a27551edc8734&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:14.321015 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791c74e-980e-4ff7-916f-043dfbf1c474 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:14.321015 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791c74e-980e-4ff7-916f-043dfbf1c474 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:14.321140 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b455d3-c72f-4fd5-8d34-8396399f458b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:14.321216 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c791c74e-980e-4ff7-916f-043dfbf1c474 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:14.321216 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c791c74e-980e-4ff7-916f-043dfbf1c474 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:14.321216 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c791c74e-980e-4ff7-916f-043dfbf1c474 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:14.337218 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 449/897] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:49:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:14.337218 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791c74e-980e-4ff7-916f-043dfbf1c474 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:14.433169 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b455d3-c72f-4fd5-8d34-8396399f458b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:14.442360 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791c74e-980e-4ff7-916f-043dfbf1c474 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:14.474822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a2868-3572-4935-b9a2-f4db091cee71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:14.475974 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a2868-3572-4935-b9a2-f4db091cee71 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:14.476445 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a2868-3572-4935-b9a2-f4db091cee71 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:14.476575 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccccf12-98c3-42a1-a29b-40401e3eede4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:14.477017 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a2868-3572-4935-b9a2-f4db091cee71 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:14.477131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccccf12-98c3-42a1-a29b-40401e3eede4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:14.477406 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccccf12-98c3-42a1-a29b-40401e3eede4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:14.477680 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccccf12-98c3-42a1-a29b-40401e3eede4 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:14.478231 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a2868-3572-4935-b9a2-f4db091cee71 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:14.478353 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccccf12-98c3-42a1-a29b-40401e3eede4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:14.479722 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccccf12-98c3-42a1-a29b-40401e3eede4 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:14.480098 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a2868-3572-4935-b9a2-f4db091cee71 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:14.481468 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:14.493563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 450/898] 10.222.0.44 () {56 vars in 1087 bytes} [Thu Jun 23 19:49:14 2022] GET /identity/v3/services => generated 1268 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 19:49:14.493563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccccf12-98c3-42a1-a29b-40401e3eede4 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:14.494437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-7b0a2868-3572-4935-b9a2-f4db091cee71 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 19:49:14.494437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:14.494437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 19:49:14.494437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 19:49:14.494437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 19:49:14.494437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 19:49:14.494437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 19:49:14.494437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:14.494437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 19:49:14.494437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:14.494437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:49:14.494437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:14.494437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:14.494437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:14.494437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:14.494437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:14.495643 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:14.495643 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:14.495643 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:14.495643 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:14.495643 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:14.495643 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:14.495643 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:14.495643 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 19:49:14.495643 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 19:49:14.495643 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 19:49:14.495643 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 19:49:14.495643 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:14.495643 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:14.495643 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:14.495643 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:14.495643 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:14.496943 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:14.496943 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:14.496943 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:14.496943 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:14.496943 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:14.496943 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:14.496943 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:14.496943 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:14.496943 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:14.496943 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:14.496943 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:14.496943 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 19:49:14.496943 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:14.496943 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 19:49:14.496943 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:14.496943 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 449/899] 10.222.0.44 () {56 vars in 1132 bytes} [Thu Jun 23 19:49:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:14.503848 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a2868-3572-4935-b9a2-f4db091cee71 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:14.545667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:14.617826 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccccf12-98c3-42a1-a29b-40401e3eede4 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:14.638783 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a2868-3572-4935-b9a2-f4db091cee71 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:14.658695 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3802a-5a94-47e3-8a0a-b88106f86a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:14.659390 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3802a-5a94-47e3-8a0a-b88106f86a0c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:14.659674 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3802a-5a94-47e3-8a0a-b88106f86a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:14.659950 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3802a-5a94-47e3-8a0a-b88106f86a0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:14.661540 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e3802a-5a94-47e3-8a0a-b88106f86a0c None admin] RBAC: Adding query filter params (role.id=ec091d6061664ed697565c76e216b5b4, scope.project.id=155180ba63c446ca889a6eb3f94cc31a, user.id=1d37f369749143ce813c693adc5f2ec7) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:14.661855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e3802a-5a94-47e3-8a0a-b88106f86a0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:14.663434 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e3802a-5a94-47e3-8a0a-b88106f86a0c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:14.676873 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 451/900] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:49:14 2022] GET /identity/v3/role_assignments?user.id=1d37f369749143ce813c693adc5f2ec7&scope.project.id=155180ba63c446ca889a6eb3f94cc31a&role.id=ec091d6061664ed697565c76e216b5b4 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:14.680415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86582117-e0e3-42b5-98b9-72b90c2dd211 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:14.680415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86582117-e0e3-42b5-98b9-72b90c2dd211 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:14.680415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86582117-e0e3-42b5-98b9-72b90c2dd211 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:14.681533 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f358b8-bcfe-4344-9169-1027c91162ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:14.682448 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f358b8-bcfe-4344-9169-1027c91162ae None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:14.682866 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f358b8-bcfe-4344-9169-1027c91162ae None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:14.683292 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f358b8-bcfe-4344-9169-1027c91162ae None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:14.684022 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f358b8-bcfe-4344-9169-1027c91162ae None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:14.684350 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f358b8-bcfe-4344-9169-1027c91162ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:14.685669 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 452/901] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:49:14.685790 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f358b8-bcfe-4344-9169-1027c91162ae None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:14.687124 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7a6c2-1329-49e8-b2d9-85f62f3a6c3e None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:14.687411 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7a6c2-1329-49e8-b2d9-85f62f3a6c3e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:14.687951 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7a6c2-1329-49e8-b2d9-85f62f3a6c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:14.692439 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 450/902] 10.222.0.44 () {56 vars in 1126 bytes} [Thu Jun 23 19:49:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:14.698687 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f358b8-bcfe-4344-9169-1027c91162ae None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:14.706269 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-cae7a6c2-1329-49e8-b2d9-85f62f3a6c3e None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:14.708015 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-cae7a6c2-1329-49e8-b2d9-85f62f3a6c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:14.786902 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:14.796206 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 453/903] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:14 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:14.819268 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7a6c2-1329-49e8-b2d9-85f62f3a6c3e None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:14.824153 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:14.834896 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f358b8-bcfe-4344-9169-1027c91162ae None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:14.879152 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c942a3-a0c3-43ec-b32c-93e8bd07b57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:14.880283 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c942a3-a0c3-43ec-b32c-93e8bd07b57a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:14.880846 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c942a3-a0c3-43ec-b32c-93e8bd07b57a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:14.881318 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c942a3-a0c3-43ec-b32c-93e8bd07b57a None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:14.882817 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c942a3-a0c3-43ec-b32c-93e8bd07b57a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:14.883652 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c942a3-a0c3-43ec-b32c-93e8bd07b57a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:14.885937 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c942a3-a0c3-43ec-b32c-93e8bd07b57a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:14.892938 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 451/904] 10.222.0.44 () {56 vars in 1126 bytes} [Thu Jun 23 19:49:14 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:14.899242 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c942a3-a0c3-43ec-b32c-93e8bd07b57a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:14.940944 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7a6c2-1329-49e8-b2d9-85f62f3a6c3e None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:14.984267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4588e-7e0c-456c-b1ac-43b0c6172c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:14.985783 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4588e-7e0c-456c-b1ac-43b0c6172c06 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:14.986513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4588e-7e0c-456c-b1ac-43b0c6172c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:14.986985 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4588e-7e0c-456c-b1ac-43b0c6172c06 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:14.987992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4588e-7e0c-456c-b1ac-43b0c6172c06 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:14.990303 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4588e-7e0c-456c-b1ac-43b0c6172c06 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:15.006757 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-71f4588e-7e0c-456c-b1ac-43b0c6172c06 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 19:49:15.006757 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:15.006757 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 19:49:15.006757 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 19:49:15.006757 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 19:49:15.006757 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 19:49:15.006757 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 19:49:15.006757 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:15.006757 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 19:49:15.006757 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:15.006757 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:49:15.006757 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:15.006757 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:15.006757 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:15.006757 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:15.006757 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:15.006757 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:15.007768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:15.007768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:15.007768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:15.007768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:15.007768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:15.007768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:15.007768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 19:49:15.007768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 19:49:15.007768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 19:49:15.007768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 19:49:15.007768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:15.007768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:15.007768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:15.007768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:15.007768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:15.007768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:15.009071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:15.009071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:15.009071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:15.009071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:15.009071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:15.009071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:15.009071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:15.009071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:15.009071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:15.009071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:15.009071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 19:49:15.009071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:15.009071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 19:49:15.009071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:15.009803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c942a3-a0c3-43ec-b32c-93e8bd07b57a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:15.017786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 454/905] 10.222.0.44 () {56 vars in 1126 bytes} [Thu Jun 23 19:49:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:15.018239 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4588e-7e0c-456c-b1ac-43b0c6172c06 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:15.048118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed84134-6878-4ce6-acec-493f7ba5c616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:15.048777 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed84134-6878-4ce6-acec-493f7ba5c616 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:15.049064 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed84134-6878-4ce6-acec-493f7ba5c616 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:15.049341 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed84134-6878-4ce6-acec-493f7ba5c616 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:15.050135 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed84134-6878-4ce6-acec-493f7ba5c616 None admin] RBAC: Adding query filter params (interface=public, service_id=fdc1ad16566b4591be3c22a4421abbff, region_id=RegionOne) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:15.050447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed84134-6878-4ce6-acec-493f7ba5c616 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:15.052042 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed84134-6878-4ce6-acec-493f7ba5c616 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:15.060955 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 452/906] 10.222.0.44 () {56 vars in 1251 bytes} [Thu Jun 23 19:49:14 2022] GET /identity/v3/endpoints?service_id=fdc1ad16566b4591be3c22a4421abbff&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:15.067322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed84134-6878-4ce6-acec-493f7ba5c616 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:15.091697 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:15.106909 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:15.143221 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4588e-7e0c-456c-b1ac-43b0c6172c06 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:15.179280 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f621f0-0f23-4762-a264-449c5a2d30b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:15.180178 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f621f0-0f23-4762-a264-449c5a2d30b7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:15.180653 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f621f0-0f23-4762-a264-449c5a2d30b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:15.181067 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f621f0-0f23-4762-a264-449c5a2d30b7 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:15.181931 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f621f0-0f23-4762-a264-449c5a2d30b7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:15.182378 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f621f0-0f23-4762-a264-449c5a2d30b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:15.183889 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f621f0-0f23-4762-a264-449c5a2d30b7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:15.189361 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 455/907] 10.222.0.44 () {56 vars in 1122 bytes} [Thu Jun 23 19:49:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:15.192364 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed84134-6878-4ce6-acec-493f7ba5c616 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:15.195760 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f621f0-0f23-4762-a264-449c5a2d30b7 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:15.227281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4ef29-c239-409b-b3c7-77ee697dba75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:15.227947 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4ef29-c239-409b-b3c7-77ee697dba75 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:15.228229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4ef29-c239-409b-b3c7-77ee697dba75 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:15.228553 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4ef29-c239-409b-b3c7-77ee697dba75 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:15.229242 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a4ef29-c239-409b-b3c7-77ee697dba75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:15.230624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a4ef29-c239-409b-b3c7-77ee697dba75 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:15.248453 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 453/908] 10.222.0.44 () {56 vars in 1087 bytes} [Thu Jun 23 19:49:15 2022] GET /identity/v3/services => generated 1268 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 19:49:15.306702 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f621f0-0f23-4762-a264-449c5a2d30b7 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:15.340822 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:15.356341 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d6ed9-b694-41e1-9b79-493128d388f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:15.356341 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d6ed9-b694-41e1-9b79-493128d388f5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:15.356341 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d6ed9-b694-41e1-9b79-493128d388f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:15.356341 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d6ed9-b694-41e1-9b79-493128d388f5 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:15.359049 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d6ed9-b694-41e1-9b79-493128d388f5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:15.359049 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d6ed9-b694-41e1-9b79-493128d388f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:15.359049 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925d6ed9-b694-41e1-9b79-493128d388f5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:15.362951 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 456/909] 10.222.0.44 () {56 vars in 1122 bytes} [Thu Jun 23 19:49:15 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:16.640322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399ca98-c9f7-4265-9b24-2c5d3417b794 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:16.640837 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399ca98-c9f7-4265-9b24-2c5d3417b794 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:16.640914 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399ca98-c9f7-4265-9b24-2c5d3417b794 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:16.642474 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 454/910] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:16.647764 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2a539-fa43-4253-b76c-a8532aedf0d3 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:16.648146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2a539-fa43-4253-b76c-a8532aedf0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:16.648328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2a539-fa43-4253-b76c-a8532aedf0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:16.665780 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-90f2a539-fa43-4253-b76c-a8532aedf0d3 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:16.666951 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-90f2a539-fa43-4253-b76c-a8532aedf0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:16.736159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 457/911] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:16 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 19:49:16.758993 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399ca98-c9f7-4265-9b24-2c5d3417b794 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:16.787019 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b3737-91b9-4975-91f4-bcf67d35f4f7 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:16.787318 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b3737-91b9-4975-91f4-bcf67d35f4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:16.787580 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b3737-91b9-4975-91f4-bcf67d35f4f7 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:16.789210 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 458/912] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:16.794400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18df835-8aa5-43b1-9d89-e2ffdc050295 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:16.794684 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18df835-8aa5-43b1-9d89-e2ffdc050295 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:16.794957 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18df835-8aa5-43b1-9d89-e2ffdc050295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:16.816439 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b18df835-8aa5-43b1-9d89-e2ffdc050295 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:16.817947 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b18df835-8aa5-43b1-9d89-e2ffdc050295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:16.862508 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:16.881376 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399ca98-c9f7-4265-9b24-2c5d3417b794 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:16.893855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 459/913] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:16 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 19:49:16.915524 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18df835-8aa5-43b1-9d89-e2ffdc050295 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:16.924904 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb23073-83cb-42f3-981d-d2c5dcc9b88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:16.926892 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb23073-83cb-42f3-981d-d2c5dcc9b88b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:16.927321 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb23073-83cb-42f3-981d-d2c5dcc9b88b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:16.927770 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb23073-83cb-42f3-981d-d2c5dcc9b88b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:16.929118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb23073-83cb-42f3-981d-d2c5dcc9b88b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:16.930971 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb23073-83cb-42f3-981d-d2c5dcc9b88b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:16.948212 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-deb23073-83cb-42f3-981d-d2c5dcc9b88b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 19:49:16.948212 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:16.948212 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 19:49:16.948212 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 19:49:16.948212 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 19:49:16.948212 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 19:49:16.948212 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 19:49:16.948212 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:16.948212 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 19:49:16.948212 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:16.948212 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:49:16.948212 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:16.948212 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:16.948212 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:16.948212 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:16.948212 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:16.948212 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:16.949439 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:16.949439 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:16.949439 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:16.949439 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:16.949439 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:16.949439 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:16.949439 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 19:49:16.949439 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 19:49:16.949439 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 19:49:16.949439 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 19:49:16.949439 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:16.949439 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:16.949439 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:16.949439 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:16.949439 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:16.949439 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:16.950456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:16.950456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:16.950456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:16.950456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:16.950456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:16.950456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:16.950456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:16.950456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:16.950456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:16.950456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:16.950456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 19:49:16.950456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:16.950456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 19:49:16.950456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:16.953561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 455/914] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jun 23 19:49:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:16.959338 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb23073-83cb-42f3-981d-d2c5dcc9b88b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:16.970558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:17.026123 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18df835-8aa5-43b1-9d89-e2ffdc050295 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:17.062197 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6532e046-c8fa-453b-8a4d-2b159d2055fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:17.063471 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532e046-c8fa-453b-8a4d-2b159d2055fb None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:17.063754 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532e046-c8fa-453b-8a4d-2b159d2055fb None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:17.064271 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532e046-c8fa-453b-8a4d-2b159d2055fb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:17.071686 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb23073-83cb-42f3-981d-d2c5dcc9b88b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:17.075698 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6532e046-c8fa-453b-8a4d-2b159d2055fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:49:17.083849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-6532e046-c8fa-453b-8a4d-2b159d2055fb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 19:49:17.083849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:17.083849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:17.083849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:17.083849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:17.083849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:17.083849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:17.083849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:17.083849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:17.083849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:17.083849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:17.083849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:17.083849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:49:17.083849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:49:17.083849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:49:17.083849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:49:17.084820 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:49:17.084820 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:49:17.084820 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:49:17.084820 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:49:17.084820 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:17.084820 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:17.084820 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:17.084820 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:17.084820 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:17.084820 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:17.084820 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:17.084820 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:17.084820 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:17.084820 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:17.084820 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:17.084820 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:17.085634 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:17.085634 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:17.085634 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:17.085634 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:17.085634 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:49:17.085634 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:49:17.085634 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:49:17.085634 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:49:17.085634 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:49:17.085634 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:49:17.085634 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 19:49:17.085634 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:17.090309 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 460/915] 10.222.0.44 () {56 vars in 1120 bytes} [Thu Jun 23 19:49:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:49:17.090922 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6532e046-c8fa-453b-8a4d-2b159d2055fb None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:17.115338 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316317cd-7b24-407b-9400-2000e62160a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:17.116467 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316317cd-7b24-407b-9400-2000e62160a1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:17.117134 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316317cd-7b24-407b-9400-2000e62160a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:17.117920 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316317cd-7b24-407b-9400-2000e62160a1 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:17.119105 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316317cd-7b24-407b-9400-2000e62160a1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:17.119609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316317cd-7b24-407b-9400-2000e62160a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:17.121897 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316317cd-7b24-407b-9400-2000e62160a1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:17.134500 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 456/916] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jun 23 19:49:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:17.135440 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316317cd-7b24-407b-9400-2000e62160a1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:17.164740 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:17.204147 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6532e046-c8fa-453b-8a4d-2b159d2055fb None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:17.211875 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:17.240337 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316317cd-7b24-407b-9400-2000e62160a1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:17.252267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae456938-8035-4de6-8dce-a0f34422614e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:17.253247 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae456938-8035-4de6-8dce-a0f34422614e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:17.254032 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae456938-8035-4de6-8dce-a0f34422614e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:17.254758 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae456938-8035-4de6-8dce-a0f34422614e None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:17.255689 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae456938-8035-4de6-8dce-a0f34422614e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:17.256405 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae456938-8035-4de6-8dce-a0f34422614e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:17.258608 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae456938-8035-4de6-8dce-a0f34422614e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:17.274026 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 461/917] 10.222.0.44 () {56 vars in 1115 bytes} [Thu Jun 23 19:49:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:17.275787 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae456938-8035-4de6-8dce-a0f34422614e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:17.284198 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7922b17-d1a9-49be-ac91-a52c4b128cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:17.285518 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7922b17-d1a9-49be-ac91-a52c4b128cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:17.285872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7922b17-d1a9-49be-ac91-a52c4b128cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:17.286210 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7922b17-d1a9-49be-ac91-a52c4b128cb5 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:17.287031 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7922b17-d1a9-49be-ac91-a52c4b128cb5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:17.287397 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7922b17-d1a9-49be-ac91-a52c4b128cb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:17.289430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7922b17-d1a9-49be-ac91-a52c4b128cb5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:17.296790 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 457/918] 10.222.0.44 () {56 vars in 1114 bytes} [Thu Jun 23 19:49:17 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:17.303165 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7922b17-d1a9-49be-ac91-a52c4b128cb5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:17.391797 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae456938-8035-4de6-8dce-a0f34422614e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:17.392804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:17.412207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7922b17-d1a9-49be-ac91-a52c4b128cb5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:17.428423 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e56cb-e14a-4de3-bd3c-01b94d6e4110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:17.429141 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e56cb-e14a-4de3-bd3c-01b94d6e4110 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:17.429415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e56cb-e14a-4de3-bd3c-01b94d6e4110 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:17.429687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e56cb-e14a-4de3-bd3c-01b94d6e4110 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:17.441215 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e56cb-e14a-4de3-bd3c-01b94d6e4110 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:17.442701 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e56cb-e14a-4de3-bd3c-01b94d6e4110 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:17.447356 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68914fb-aef7-4558-8869-ce9f9dece55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:17.449327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68914fb-aef7-4558-8869-ce9f9dece55c None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:17.449327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68914fb-aef7-4558-8869-ce9f9dece55c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:17.449327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68914fb-aef7-4558-8869-ce9f9dece55c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:17.449327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68914fb-aef7-4558-8869-ce9f9dece55c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:17.450662 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68914fb-aef7-4558-8869-ce9f9dece55c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:17.460023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-152e56cb-e14a-4de3-bd3c-01b94d6e4110 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:17.460023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:17.460023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:17.460023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:17.460023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:17.460023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:17.460023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:17.460023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:17.460023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:17.460023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:17.460023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:17.460023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:17.460023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:49:17.460023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:49:17.460023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:49:17.461152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:49:17.461152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:17.461152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:17.461152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:17.461152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:17.461152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:17.461152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:17.461152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:17.461152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:17.461152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:17.461152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:17.461152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:17.461152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:17.461152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:17.461152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:17.461152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:17.461152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:17.462380 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:49:17.462380 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:49:17.462380 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:17.462380 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:17.462380 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 462/919] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:17.462738 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372fe43-568a-4b77-a1f9-2d1d4c817354 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:17.462855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372fe43-568a-4b77-a1f9-2d1d4c817354 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:17.463126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372fe43-568a-4b77-a1f9-2d1d4c817354 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:17.464373 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 463/920] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:17.465404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e438a-aa0c-4898-a4dd-6fff3cfac136 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:17.465829 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e438a-aa0c-4898-a4dd-6fff3cfac136 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:17.465944 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e438a-aa0c-4898-a4dd-6fff3cfac136 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:17.467119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 464/921] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:17.467839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e438a-aa0c-4898-a4dd-6fff3cfac136 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:17.490097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:17.529681 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 458/922] 10.222.0.44 () {60 vars in 1144 bytes} [Thu Jun 23 19:49:17 2022] POST /identity/v3/endpoints => generated 346 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:49:17.531010 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d2f41-2ecc-40fc-b1cf-3215ec3424d3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:17.531304 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d2f41-2ecc-40fc-b1cf-3215ec3424d3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:17.531588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d2f41-2ecc-40fc-b1cf-3215ec3424d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:17.557191 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-824d2f41-2ecc-40fc-b1cf-3215ec3424d3 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:17.558285 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-824d2f41-2ecc-40fc-b1cf-3215ec3424d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:17.580704 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e438a-aa0c-4898-a4dd-6fff3cfac136 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:17.620994 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4710535-fa16-44a3-a372-bbf0779c2133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:17.621664 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4710535-fa16-44a3-a372-bbf0779c2133 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:17.621951 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4710535-fa16-44a3-a372-bbf0779c2133 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:17.622207 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4710535-fa16-44a3-a372-bbf0779c2133 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:17.622938 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4710535-fa16-44a3-a372-bbf0779c2133 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:17.623242 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4710535-fa16-44a3-a372-bbf0779c2133 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:17.624670 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4710535-fa16-44a3-a372-bbf0779c2133 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:17.642056 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 465/923] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:17.642266 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 459/924] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:17 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:17.642725 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9d168-262c-4c06-a87d-a67810f8e127 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:17.642725 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9d168-262c-4c06-a87d-a67810f8e127 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:17.642725 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9d168-262c-4c06-a87d-a67810f8e127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:17.643521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d2f41-2ecc-40fc-b1cf-3215ec3424d3 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:17.660173 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8fd9d168-262c-4c06-a87d-a67810f8e127 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:17.661420 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8fd9d168-262c-4c06-a87d-a67810f8e127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:17.723921 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:17.743434 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 466/925] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:17 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:17.744278 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9d168-262c-4c06-a87d-a67810f8e127 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:17.755235 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:17.769727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d2f41-2ecc-40fc-b1cf-3215ec3424d3 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:17.806158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d80ce5-206a-46e1-9c85-af1dd1126f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:17.806980 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d80ce5-206a-46e1-9c85-af1dd1126f81 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:17.807264 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d80ce5-206a-46e1-9c85-af1dd1126f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:17.807538 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d80ce5-206a-46e1-9c85-af1dd1126f81 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:17.815258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d80ce5-206a-46e1-9c85-af1dd1126f81 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:17.817372 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d80ce5-206a-46e1-9c85-af1dd1126f81 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:17.835968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-79d80ce5-206a-46e1-9c85-af1dd1126f81 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 19:49:17.835968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:17.835968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:17.835968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:17.835968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:17.835968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:17.835968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:17.835968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:17.835968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:17.835968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:17.835968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:17.835968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:17.835968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:49:17.835968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:49:17.835968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:49:17.835968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:49:17.836981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:17.836981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:17.836981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:49:17.836981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:49:17.836981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:49:17.836981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:49:17.836981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:17.836981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:17.836981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:17.836981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:17.836981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:17.836981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:17.836981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:17.836981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:17.836981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:17.836981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:17.836981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:17.836981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:17.837953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:17.837953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:17.837953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:49:17.837953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:49:17.837953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:49:17.837953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:49:17.837953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:49:17.837953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:49:17.837953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 19:49:17.837953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:17.837953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 460/926] 10.222.0.44 () {56 vars in 1117 bytes} [Thu Jun 23 19:49:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:17.841118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d80ce5-206a-46e1-9c85-af1dd1126f81 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:17.855338 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9d168-262c-4c06-a87d-a67810f8e127 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:17.892810 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf30c93-2918-4da0-97b0-31157d3df228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:17.893630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf30c93-2918-4da0-97b0-31157d3df228 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:17.893910 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf30c93-2918-4da0-97b0-31157d3df228 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:17.894184 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf30c93-2918-4da0-97b0-31157d3df228 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:17.894878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf30c93-2918-4da0-97b0-31157d3df228 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:17.896265 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf30c93-2918-4da0-97b0-31157d3df228 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:17.909112 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-7cf30c93-2918-4da0-97b0-31157d3df228 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 19:49:17.909112 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:17.909112 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 19:49:17.909112 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 19:49:17.909112 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 19:49:17.909112 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 19:49:17.909112 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 19:49:17.909112 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:17.909112 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 19:49:17.909112 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:17.909112 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:49:17.909112 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:17.909112 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:17.909112 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:17.909112 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:17.909112 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:17.910177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:17.910177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:17.910177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:17.910177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:17.910177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:17.910177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:17.910177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:17.910177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 19:49:17.910177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 19:49:17.910177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 19:49:17.910177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 19:49:17.910177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:17.910177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:17.910177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:17.910177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:17.910177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:17.911023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:17.911023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:17.911023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:17.911023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:17.911023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:17.911023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:17.911023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:17.911023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:17.911023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:17.911023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:17.911023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:17.911023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 19:49:17.911023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:17.911023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 19:49:17.911023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:17.911023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 467/927] 10.222.0.44 () {56 vars in 1132 bytes} [Thu Jun 23 19:49:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:17.911865 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf30c93-2918-4da0-97b0-31157d3df228 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:17.950185 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d80ce5-206a-46e1-9c85-af1dd1126f81 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:17.977594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:17.986127 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da882fcd-99f8-41b9-82ab-a1b846ee12a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:17.986789 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da882fcd-99f8-41b9-82ab-a1b846ee12a7 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:17.987069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da882fcd-99f8-41b9-82ab-a1b846ee12a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:17.987391 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da882fcd-99f8-41b9-82ab-a1b846ee12a7 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:17.994841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da882fcd-99f8-41b9-82ab-a1b846ee12a7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:17.994841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da882fcd-99f8-41b9-82ab-a1b846ee12a7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:17.995043 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:18.020781 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf30c93-2918-4da0-97b0-31157d3df228 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:18.053142 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b896d4cd-5e42-4aad-a3c7-f0aaf8b1abb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:18.053807 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896d4cd-5e42-4aad-a3c7-f0aaf8b1abb5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:18.054091 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896d4cd-5e42-4aad-a3c7-f0aaf8b1abb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:18.054366 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b896d4cd-5e42-4aad-a3c7-f0aaf8b1abb5 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:18.055245 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b896d4cd-5e42-4aad-a3c7-f0aaf8b1abb5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:18.055555 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b896d4cd-5e42-4aad-a3c7-f0aaf8b1abb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:18.057030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b896d4cd-5e42-4aad-a3c7-f0aaf8b1abb5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:18.074949 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 468/928] 10.222.0.44 () {56 vars in 1149 bytes} [Thu Jun 23 19:49:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:18.075805 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 461/929] 10.222.0.44 () {60 vars in 1141 bytes} [Thu Jun 23 19:49:17 2022] POST /identity/v3/services => generated 241 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:49:18.076142 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b896d4cd-5e42-4aad-a3c7-f0aaf8b1abb5 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:18.080209 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da882fcd-99f8-41b9-82ab-a1b846ee12a7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:18.193932 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b896d4cd-5e42-4aad-a3c7-f0aaf8b1abb5 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:18.224238 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:18.228085 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da882fcd-99f8-41b9-82ab-a1b846ee12a7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:18.244879 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce28be-a6f3-4322-9e0b-08973dc5951e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:18.245694 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce28be-a6f3-4322-9e0b-08973dc5951e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:18.246052 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce28be-a6f3-4322-9e0b-08973dc5951e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:18.246669 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce28be-a6f3-4322-9e0b-08973dc5951e None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:18.247486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce28be-a6f3-4322-9e0b-08973dc5951e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:18.248470 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce28be-a6f3-4322-9e0b-08973dc5951e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:18.249934 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce28be-a6f3-4322-9e0b-08973dc5951e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:18.258062 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 469/930] 10.222.0.44 () {56 vars in 1126 bytes} [Thu Jun 23 19:49:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:18.262146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce28be-a6f3-4322-9e0b-08973dc5951e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:18.277450 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f09f9-79f0-4069-8c1e-cec04d6e3a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:18.278250 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f09f9-79f0-4069-8c1e-cec04d6e3a43 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:18.278538 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f09f9-79f0-4069-8c1e-cec04d6e3a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:18.278812 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f09f9-79f0-4069-8c1e-cec04d6e3a43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:18.290873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f09f9-79f0-4069-8c1e-cec04d6e3a43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:18.292506 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f09f9-79f0-4069-8c1e-cec04d6e3a43 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:18.314047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-458f09f9-79f0-4069-8c1e-cec04d6e3a43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:18.314047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:18.314047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:18.314047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:18.314047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:18.314047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:18.314047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:18.314047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:18.314047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:18.314047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:18.314047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:18.314047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:18.314047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:49:18.314047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:49:18.314047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:49:18.314994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:49:18.314994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:18.314994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:18.314994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:18.314994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:18.314994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:18.314994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:18.314994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:18.314994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:18.314994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:18.314994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:18.314994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:18.314994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:18.314994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:18.314994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:18.314994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:18.314994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:18.315869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:49:18.315869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:49:18.315869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:18.315869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:18.317167 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 462/931] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:18.325835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f09f9-79f0-4069-8c1e-cec04d6e3a43 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:18.361814 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:18.395221 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce28be-a6f3-4322-9e0b-08973dc5951e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:18.432190 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f09f9-79f0-4069-8c1e-cec04d6e3a43 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:18.439420 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33401603-b6db-4071-9d74-2f6238e9418c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:18.440269 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33401603-b6db-4071-9d74-2f6238e9418c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:18.440766 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33401603-b6db-4071-9d74-2f6238e9418c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:18.441189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33401603-b6db-4071-9d74-2f6238e9418c None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:18.442600 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33401603-b6db-4071-9d74-2f6238e9418c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:18.443058 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33401603-b6db-4071-9d74-2f6238e9418c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:18.444988 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33401603-b6db-4071-9d74-2f6238e9418c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:18.452118 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 470/932] 10.222.0.44 () {56 vars in 1126 bytes} [Thu Jun 23 19:49:18 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:18.457749 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33401603-b6db-4071-9d74-2f6238e9418c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:18.471781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d91b1b-fefd-4794-be34-5eb373860a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:18.472918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d91b1b-fefd-4794-be34-5eb373860a9e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:18.473501 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d91b1b-fefd-4794-be34-5eb373860a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:18.474027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d91b1b-fefd-4794-be34-5eb373860a9e None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:18.474951 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d91b1b-fefd-4794-be34-5eb373860a9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:18.475480 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d91b1b-fefd-4794-be34-5eb373860a9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:18.477085 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d91b1b-fefd-4794-be34-5eb373860a9e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:18.494438 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 463/933] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:18.495583 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d91b1b-fefd-4794-be34-5eb373860a9e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:18.523407 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:18.570065 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33401603-b6db-4071-9d74-2f6238e9418c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:18.612235 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:18.616486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77187d7-95e9-47cc-8cd5-e94cb690ff5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:18.622952 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77187d7-95e9-47cc-8cd5-e94cb690ff5b None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:18.622952 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77187d7-95e9-47cc-8cd5-e94cb690ff5b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:18.622952 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77187d7-95e9-47cc-8cd5-e94cb690ff5b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:18.622952 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77187d7-95e9-47cc-8cd5-e94cb690ff5b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:18.622952 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77187d7-95e9-47cc-8cd5-e94cb690ff5b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:18.623336 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d91b1b-fefd-4794-be34-5eb373860a9e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:18.658789 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ac2a1-44a7-40e7-8e4b-bf7f288bbff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:18.659760 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ac2a1-44a7-40e7-8e4b-bf7f288bbff1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:18.660203 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ac2a1-44a7-40e7-8e4b-bf7f288bbff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:18.660576 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ac2a1-44a7-40e7-8e4b-bf7f288bbff1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:18.668710 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ac2a1-44a7-40e7-8e4b-bf7f288bbff1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:18.670206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ac2a1-44a7-40e7-8e4b-bf7f288bbff1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:18.685960 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-c10ac2a1-44a7-40e7-8e4b-bf7f288bbff1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:49:18.685960 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:18.685960 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:18.685960 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:18.685960 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:18.685960 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:18.685960 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:18.685960 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:18.685960 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:18.685960 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:18.685960 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:18.685960 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:18.685960 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:49:18.685960 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:49:18.685960 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:49:18.686949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:49:18.686949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:18.686949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:18.686949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:18.686949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:18.686949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:18.686949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:18.686949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:18.686949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:18.686949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:18.686949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:18.686949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:18.686949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:18.686949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:18.686949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:18.686949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:18.686949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:18.688134 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:49:18.688134 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:49:18.688134 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:49:18.688134 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:49:18.688134 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:49:18.688134 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:49:18.688134 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:49:18.688134 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:18.691946 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 464/934] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:49:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:18.693799 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ac2a1-44a7-40e7-8e4b-bf7f288bbff1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:18.705550 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 471/935] 10.222.0.44 () {60 vars in 1144 bytes} [Thu Jun 23 19:49:18 2022] POST /identity/v3/endpoints => generated 347 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:49:18.820805 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ac2a1-44a7-40e7-8e4b-bf7f288bbff1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:18.864008 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81559b-a586-491e-9b81-fb989824c625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:18.865012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81559b-a586-491e-9b81-fb989824c625 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:18.865463 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81559b-a586-491e-9b81-fb989824c625 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:18.865887 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81559b-a586-491e-9b81-fb989824c625 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:18.866830 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81559b-a586-491e-9b81-fb989824c625 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:18.867293 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81559b-a586-491e-9b81-fb989824c625 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:18.868944 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81559b-a586-491e-9b81-fb989824c625 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:18.883390 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 465/936] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:49:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:18.889624 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77187d7-95e9-47cc-8cd5-e94cb690ff5b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:18.993261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77187d7-95e9-47cc-8cd5-e94cb690ff5b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:19.006868 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:19.028589 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb59d43b-4715-45ca-9ba3-3934620b1a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:19.034819 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59d43b-4715-45ca-9ba3-3934620b1a6d None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:19.034819 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59d43b-4715-45ca-9ba3-3934620b1a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:19.034819 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb59d43b-4715-45ca-9ba3-3934620b1a6d None admin] PATH_INFO: `/v3/projects/155180ba63c446ca889a6eb3f94cc31a/users/1d37f369749143ce813c693adc5f2ec7/roles/ec091d6061664ed697565c76e216b5b4` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:19.034819 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59d43b-4715-45ca-9ba3-3934620b1a6d None admin] RBAC: Authorizing `identity:create_grant(project_id=155180ba63c446ca889a6eb3f94cc31a, user_id=1d37f369749143ce813c693adc5f2ec7, role_id=ec091d6061664ed697565c76e216b5b4)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:19.034819 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb59d43b-4715-45ca-9ba3-3934620b1a6d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:19.114520 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 472/937] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jun 23 19:49:18 2022] PUT /identity/v3/projects/155180ba63c446ca889a6eb3f94cc31a/users/1d37f369749143ce813c693adc5f2ec7/roles/ec091d6061664ed697565c76e216b5b4 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:49:19.602027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae76df-4541-4fc9-9de8-17e5d6bd07a4 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:19.602427 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae76df-4541-4fc9-9de8-17e5d6bd07a4 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:19.602613 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae76df-4541-4fc9-9de8-17e5d6bd07a4 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:19.604157 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 466/938] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:19.609437 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd176a6c-abb7-4f42-98e0-873e691f5870 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:19.609734 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd176a6c-abb7-4f42-98e0-873e691f5870 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:19.610014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd176a6c-abb7-4f42-98e0-873e691f5870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:19.626852 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-cd176a6c-abb7-4f42-98e0-873e691f5870 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:19.628058 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-cd176a6c-abb7-4f42-98e0-873e691f5870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:19.697784 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 473/939] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:19.721660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae76df-4541-4fc9-9de8-17e5d6bd07a4 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:19.734942 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:19.828609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae76df-4541-4fc9-9de8-17e5d6bd07a4 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:19.865240 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549660b-d81c-4361-91d3-044ab19c0150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:19.866238 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549660b-d81c-4361-91d3-044ab19c0150 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:19.866771 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549660b-d81c-4361-91d3-044ab19c0150 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:19.867239 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549660b-d81c-4361-91d3-044ab19c0150 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:19.868193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d549660b-d81c-4361-91d3-044ab19c0150 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:19.869857 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d549660b-d81c-4361-91d3-044ab19c0150 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:19.871531 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: WARNING keystone.server.flask.common [None req-d549660b-d81c-4361-91d3-044ab19c0150 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 23 19:49:19.923764 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.sql.core [None req-d549660b-d81c-4361-91d3-044ab19c0150 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 19:49:19.923764 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 23 19:49:19.923764 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [parameters: {'id': '2483f18e23234673820bb511290d0430', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 23 19:49:19.923764 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80363) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 19:49:19.932469 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-d549660b-d81c-4361-91d3-044ab19c0150 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2483f18e23234673820bb511290d0430.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2483f18e23234673820bb511290d0430. Jun 23 19:49:19.932469 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:19.932469 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 19:49:19.932469 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 19:49:19.932469 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 19:49:19.932469 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 19:49:19.932469 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 19:49:19.932469 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 19:49:19.932469 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 19:49:19.932469 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application conn.query(q) Jun 23 19:49:19.932469 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 19:49:19.932469 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 19:49:19.932469 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 19:49:19.938651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result.read() Jun 23 19:49:19.938651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 19:49:19.938651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 19:49:19.938651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 19:49:19.938651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 19:49:19.938651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 19:49:19.938651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 19:49:19.938651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 19:49:19.938651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 19:49:19.938651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 19:49:19.938651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:19.938651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 19:49:19.938651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:19.938651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:19.938651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 19:49:19.938651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 19:49:19.938651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 23 19:49:19.938651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 23 19:49:19.939591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 19:49:19.939591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application next(self.gen) Jun 23 19:49:19.939591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 19:49:19.939591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application yield resource Jun 23 19:49:19.939591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 19:49:19.939591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application next(self.gen) Jun 23 19:49:19.939591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 19:49:19.939591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 19:49:19.939591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 19:49:19.939591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 19:49:19.939591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 19:49:19.939591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise self.value Jun 23 19:49:19.939591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 19:49:19.939591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 19:49:19.939591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 19:49:19.939591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application session.commit() Jun 23 19:49:19.939591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 19:49:19.940517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 19:49:19.940517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 19:49:19.940517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 19:49:19.940517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 19:49:19.940517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.session.flush() Jun 23 19:49:19.940517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 19:49:19.940517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 19:49:19.940517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 19:49:19.940517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 19:49:19.940517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 19:49:19.940517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application compat.raise_( Jun 23 19:49:19.940517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 19:49:19.940517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception Jun 23 19:49:19.940517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 19:49:19.940517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 19:49:19.940517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 19:49:19.940517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 19:49:19.941458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 19:49:19.941458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 19:49:19.941458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 19:49:19.941458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 19:49:19.941458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 23 19:49:19.941458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 23 19:49:19.941458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 19:49:19.941458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 19:49:19.941458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 19:49:19.941458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 19:49:19.941458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 19:49:19.941458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 19:49:19.941458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 19:49:19.941458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 19:49:19.941458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 19:49:19.942244 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 19:49:19.942244 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 19:49:19.942244 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception Jun 23 19:49:19.942244 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 19:49:19.942244 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 19:49:19.942244 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 19:49:19.942244 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 19:49:19.942244 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 19:49:19.942244 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 19:49:19.942244 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 19:49:19.942244 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application conn.query(q) Jun 23 19:49:19.942244 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 19:49:19.942244 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 19:49:19.942244 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 19:49:19.942244 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result.read() Jun 23 19:49:19.942244 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 19:49:19.942244 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 19:49:19.943108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 19:49:19.943108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 19:49:19.943108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 19:49:19.943108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 19:49:19.943108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 19:49:19.943108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 19:49:19.943108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 19:49:19.943108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 23 19:49:19.943108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [parameters: {'id': '2483f18e23234673820bb511290d0430', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 23 19:49:19.943108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 19:49:19.943108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:19.943108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:49:19.943108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:19.943108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:19.943108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 23 19:49:19.943889 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 23 19:49:19.943889 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 19:49:19.943889 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 19:49:19.943889 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 23 19:49:19.943889 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:19.943889 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:49:19.943889 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:19.943889 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:19.943889 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:19.943889 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:19.943889 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:19.943889 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:19.943889 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:19.943889 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:19.943889 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:19.943889 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:19.943889 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:19.948968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:19.948968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 23 19:49:19.948968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 23 19:49:19.948968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:19.948968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:19.948968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 23 19:49:19.948968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 23 19:49:19.948968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2483f18e23234673820bb511290d0430. Jun 23 19:49:19.948968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:19.948968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 467/940] 10.222.0.44 () {60 vars in 1140 bytes} [Thu Jun 23 19:49:19 2022] POST /identity/v3/projects => generated 262 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:19.950249 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd176a6c-abb7-4f42-98e0-873e691f5870 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:20.053912 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd176a6c-abb7-4f42-98e0-873e691f5870 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:20.055427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:20.089008 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad32c1-99f1-4bf8-8670-d154f23b2486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:20.089809 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad32c1-99f1-4bf8-8670-d154f23b2486 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:20.090095 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad32c1-99f1-4bf8-8670-d154f23b2486 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:20.090375 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad32c1-99f1-4bf8-8670-d154f23b2486 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:20.097222 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faad32c1-99f1-4bf8-8670-d154f23b2486 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:20.098758 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faad32c1-99f1-4bf8-8670-d154f23b2486 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:20.112426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-faad32c1-99f1-4bf8-8670-d154f23b2486 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:49:20.112426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:20.112426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:20.112426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:20.112426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:20.112426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:20.112426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:20.112426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:20.112426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:20.112426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:20.112426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:20.112426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:20.112426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:49:20.112426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:49:20.112426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:49:20.113839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:49:20.113839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:20.113839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:20.113839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:20.113839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:20.113839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:20.113839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:20.113839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:20.113839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:20.113839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:20.113839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:20.113839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:20.113839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:20.113839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:20.113839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:20.113839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:20.113839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:20.114890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:49:20.114890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:49:20.114890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:49:20.114890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:49:20.114890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:49:20.114890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:49:20.114890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:49:20.114890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:20.114890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 474/941] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:49:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:20.120247 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549660b-d81c-4361-91d3-044ab19c0150 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:20.191723 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:20.231913 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549660b-d81c-4361-91d3-044ab19c0150 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:20.240131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0837956-fb4c-4071-ace8-f198649491f2 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:20.240531 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0837956-fb4c-4071-ace8-f198649491f2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:20.240889 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0837956-fb4c-4071-ace8-f198649491f2 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:20.242586 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 475/942] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:20.247586 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da9d8c-647d-4b3a-808a-f869613deff3 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:20.247980 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da9d8c-647d-4b3a-808a-f869613deff3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:20.248349 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da9d8c-647d-4b3a-808a-f869613deff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:20.268808 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080a114-b140-413e-9003-10b7f4340e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:20.270000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-88da9d8c-647d-4b3a-808a-f869613deff3 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:20.270517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080a114-b140-413e-9003-10b7f4340e7b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:20.271088 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080a114-b140-413e-9003-10b7f4340e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:20.271401 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-88da9d8c-647d-4b3a-808a-f869613deff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:20.271577 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080a114-b140-413e-9003-10b7f4340e7b None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:20.273113 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080a114-b140-413e-9003-10b7f4340e7b None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:20.273671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080a114-b140-413e-9003-10b7f4340e7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:20.276162 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080a114-b140-413e-9003-10b7f4340e7b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:20.298340 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 468/943] 10.222.0.44 () {56 vars in 1112 bytes} [Thu Jun 23 19:49:20 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:20.352513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 476/944] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:20.377064 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080a114-b140-413e-9003-10b7f4340e7b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:20.490268 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080a114-b140-413e-9003-10b7f4340e7b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:20.518326 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:20.527499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40387eb-0176-425a-80e0-95f19c128511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:20.528323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40387eb-0176-425a-80e0-95f19c128511 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:20.528605 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40387eb-0176-425a-80e0-95f19c128511 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:20.528870 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40387eb-0176-425a-80e0-95f19c128511 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:20.529568 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40387eb-0176-425a-80e0-95f19c128511 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:20.531049 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40387eb-0176-425a-80e0-95f19c128511 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:20.543617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-e40387eb-0176-425a-80e0-95f19c128511 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 19:49:20.543617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:20.543617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 19:49:20.543617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 19:49:20.543617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 19:49:20.543617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 19:49:20.543617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 19:49:20.543617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:20.543617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 19:49:20.543617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:20.543617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:49:20.543617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:20.543617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:20.543617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:20.543617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:20.543617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:20.543617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:20.544670 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:20.544670 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:20.544670 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:20.544670 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:20.544670 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:20.544670 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:20.544670 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 19:49:20.544670 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 19:49:20.544670 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 19:49:20.544670 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 19:49:20.544670 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:20.544670 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:20.544670 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:20.544670 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:20.544670 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:20.544670 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:20.545507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:20.545507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:20.545507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:20.545507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:20.545507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:20.545507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:20.545507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:20.545507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:20.545507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:20.545507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:20.545507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 19:49:20.545507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:20.545507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 19:49:20.545507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:20.545507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 469/945] 10.222.0.44 () {56 vars in 1126 bytes} [Thu Jun 23 19:49:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:20.552923 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da9d8c-647d-4b3a-808a-f869613deff3 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:20.654209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:20.664076 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da9d8c-647d-4b3a-808a-f869613deff3 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:20.700631 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6f38f-5d32-43f8-8d3c-8a7a440db0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:20.701545 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6f38f-5d32-43f8-8d3c-8a7a440db0ae None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:20.701969 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6f38f-5d32-43f8-8d3c-8a7a440db0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:20.702393 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6f38f-5d32-43f8-8d3c-8a7a440db0ae None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:20.703268 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d6f38f-5d32-43f8-8d3c-8a7a440db0ae None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:20.703741 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d6f38f-5d32-43f8-8d3c-8a7a440db0ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:20.705426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d6f38f-5d32-43f8-8d3c-8a7a440db0ae None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:20.711347 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 477/946] 10.222.0.44 () {56 vars in 1122 bytes} [Thu Jun 23 19:49:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:20.717103 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40387eb-0176-425a-80e0-95f19c128511 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:20.826391 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40387eb-0176-425a-80e0-95f19c128511 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:20.863534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb9cb6-ee10-4a9d-93b6-fca2e05de58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:20.864482 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb9cb6-ee10-4a9d-93b6-fca2e05de58b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:20.864917 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb9cb6-ee10-4a9d-93b6-fca2e05de58b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:20.865341 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb9cb6-ee10-4a9d-93b6-fca2e05de58b None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:20.866192 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb9cb6-ee10-4a9d-93b6-fca2e05de58b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:20.866728 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb9cb6-ee10-4a9d-93b6-fca2e05de58b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:20.868243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb9cb6-ee10-4a9d-93b6-fca2e05de58b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:20.874195 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 470/947] 10.222.0.44 () {56 vars in 1122 bytes} [Thu Jun 23 19:49:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:20.880162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6f38f-5d32-43f8-8d3c-8a7a440db0ae None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:20.895832 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5ef19-712d-4b2c-a1e4-38361f5c8594 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:20.896189 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5ef19-712d-4b2c-a1e4-38361f5c8594 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:20.896443 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5ef19-712d-4b2c-a1e4-38361f5c8594 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:20.897815 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 471/948] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:49:20.903880 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edcb107-b50d-4e00-aa36-22a388bbb1f4 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:20.904175 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edcb107-b50d-4e00-aa36-22a388bbb1f4 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:20.904524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edcb107-b50d-4e00-aa36-22a388bbb1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:20.922122 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1edcb107-b50d-4e00-aa36-22a388bbb1f4 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:20.923650 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1edcb107-b50d-4e00-aa36-22a388bbb1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:20.977297 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:20.991214 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6f38f-5d32-43f8-8d3c-8a7a440db0ae None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:21.004321 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 472/949] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 19:49:21.018493 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edcb107-b50d-4e00-aa36-22a388bbb1f4 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:21.036526 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dbfda6-1ebd-479b-aa6e-552c44fa0824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:21.037596 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbfda6-1ebd-479b-aa6e-552c44fa0824 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:21.038080 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbfda6-1ebd-479b-aa6e-552c44fa0824 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:21.038555 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dbfda6-1ebd-479b-aa6e-552c44fa0824 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:21.039667 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dbfda6-1ebd-479b-aa6e-552c44fa0824 None admin] RBAC: Adding query filter params (interface=public, service_id=21cbf623842f4ea5bd11af6c3805be3f, region_id=RegionOne) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:21.040180 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dbfda6-1ebd-479b-aa6e-552c44fa0824 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:21.041830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dbfda6-1ebd-479b-aa6e-552c44fa0824 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:21.050133 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 478/950] 10.222.0.44 () {56 vars in 1251 bytes} [Thu Jun 23 19:49:20 2022] GET /identity/v3/endpoints?service_id=21cbf623842f4ea5bd11af6c3805be3f&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:21.055608 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dbfda6-1ebd-479b-aa6e-552c44fa0824 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:21.093423 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:21.145770 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edcb107-b50d-4e00-aa36-22a388bbb1f4 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:21.169564 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dbfda6-1ebd-479b-aa6e-552c44fa0824 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:21.196326 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e8146-54c9-4b78-ab43-93a3ecfbed1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:21.196326 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e8146-54c9-4b78-ab43-93a3ecfbed1f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:21.196326 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e8146-54c9-4b78-ab43-93a3ecfbed1f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:21.196326 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e8146-54c9-4b78-ab43-93a3ecfbed1f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:21.196785 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e8146-54c9-4b78-ab43-93a3ecfbed1f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:21.196785 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e8146-54c9-4b78-ab43-93a3ecfbed1f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:21.208624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-0c8e8146-54c9-4b78-ab43-93a3ecfbed1f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 19:49:21.208624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:21.208624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 19:49:21.208624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 19:49:21.208624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 19:49:21.208624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 19:49:21.208624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 19:49:21.208624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:21.208624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 19:49:21.208624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:21.208624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:49:21.208624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:21.208624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:21.208624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:21.208624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:21.208624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:21.208624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:21.210072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:21.210072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:21.210072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:21.210072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:21.210072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:21.210072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:21.210072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 19:49:21.210072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 19:49:21.210072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 19:49:21.210072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 19:49:21.210072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:21.210072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:21.210072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:21.210072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:21.210072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:21.210072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:21.210968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:21.210968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:21.210968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:21.210968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:21.210968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:21.210968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:21.210968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:21.210968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:21.210968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:21.210968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:21.210968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 19:49:21.210968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:21.210968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 19:49:21.210968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:21.216115 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 473/951] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:49:21 2022] GET /identity/v3/services/compute => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:49:21.217141 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e8146-54c9-4b78-ab43-93a3ecfbed1f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:21.219660 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30878275-d666-489d-8a81-5aafe6a7a1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:21.220545 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30878275-d666-489d-8a81-5aafe6a7a1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:21.221013 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30878275-d666-489d-8a81-5aafe6a7a1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:21.221288 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30878275-d666-489d-8a81-5aafe6a7a1b1 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:21.221967 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30878275-d666-489d-8a81-5aafe6a7a1b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:21.223897 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30878275-d666-489d-8a81-5aafe6a7a1b1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:21.230466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 479/952] 10.222.0.44 () {56 vars in 1087 bytes} [Thu Jun 23 19:49:21 2022] GET /identity/v3/services => generated 1497 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 19:49:21.274989 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:21.282108 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27db5d3-a751-4324-87bd-ce76df449eff None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:21.282445 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27db5d3-a751-4324-87bd-ce76df449eff None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:21.282678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27db5d3-a751-4324-87bd-ce76df449eff None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:21.285912 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 480/953] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:49:21.290425 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f39463-9bf7-488a-90f1-84770aafef3f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:21.290716 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f39463-9bf7-488a-90f1-84770aafef3f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:21.290994 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f39463-9bf7-488a-90f1-84770aafef3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:21.311570 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-98f39463-9bf7-488a-90f1-84770aafef3f None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:21.315514 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-98f39463-9bf7-488a-90f1-84770aafef3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:21.337938 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e8146-54c9-4b78-ab43-93a3ecfbed1f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:21.381913 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a9af7-a945-4279-b790-7a3c73583b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:21.382602 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a9af7-a945-4279-b790-7a3c73583b5c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:21.383423 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a9af7-a945-4279-b790-7a3c73583b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:21.383713 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a9af7-a945-4279-b790-7a3c73583b5c None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:21.384529 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a9af7-a945-4279-b790-7a3c73583b5c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:21.384841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a9af7-a945-4279-b790-7a3c73583b5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:21.390832 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a9af7-a945-4279-b790-7a3c73583b5c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:21.393691 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 474/954] 10.222.0.44 () {56 vars in 1112 bytes} [Thu Jun 23 19:49:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:21.399296 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a9af7-a945-4279-b790-7a3c73583b5c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:21.407409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 481/955] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:21 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:21.434012 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f39463-9bf7-488a-90f1-84770aafef3f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:21.453919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:21.520925 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a9af7-a945-4279-b790-7a3c73583b5c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:21.535410 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:21.565656 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9fa97e-f2cc-4d78-9133-e342d901fffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:21.566383 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fa97e-f2cc-4d78-9133-e342d901fffc None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:21.566668 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fa97e-f2cc-4d78-9133-e342d901fffc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:21.566944 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fa97e-f2cc-4d78-9133-e342d901fffc None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:21.567653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9fa97e-f2cc-4d78-9133-e342d901fffc None admin] RBAC: Adding query filter params (type=compute) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:21.568053 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9fa97e-f2cc-4d78-9133-e342d901fffc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:21.570348 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f39463-9bf7-488a-90f1-84770aafef3f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:21.573015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9fa97e-f2cc-4d78-9133-e342d901fffc None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:21.578355 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 475/956] 10.222.0.44 () {56 vars in 1112 bytes} [Thu Jun 23 19:49:21 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:21.613920 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de8e03-72bd-4037-8696-d9a44112b880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:21.615039 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de8e03-72bd-4037-8696-d9a44112b880 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:21.615524 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de8e03-72bd-4037-8696-d9a44112b880 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:21.615942 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5de8e03-72bd-4037-8696-d9a44112b880 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:21.628284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5de8e03-72bd-4037-8696-d9a44112b880 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:49:21.637217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-f5de8e03-72bd-4037-8696-d9a44112b880 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 19:49:21.637217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:21.637217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:21.637217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:21.637217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:21.637217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:21.637217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:21.637217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:21.637217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:21.637217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:21.637217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:21.637217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:21.637217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:49:21.637217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:49:21.637217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:49:21.637217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:49:21.638455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:49:21.638455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:49:21.638455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:49:21.638455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:49:21.638455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:21.638455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:21.638455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:21.638455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:21.638455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:21.638455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:21.638455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:21.638455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:21.638455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:21.638455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:21.638455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:21.638455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:21.639598 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:21.639598 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:21.639598 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:21.639598 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:21.639598 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:49:21.639598 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:49:21.639598 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:49:21.639598 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:49:21.639598 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:49:21.639598 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:49:21.639598 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 19:49:21.639598 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:21.647450 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 482/957] 10.222.0.44 () {56 vars in 1120 bytes} [Thu Jun 23 19:49:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:21.649162 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9fa97e-f2cc-4d78-9133-e342d901fffc None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:21.760195 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9fa97e-f2cc-4d78-9133-e342d901fffc None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:21.796056 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19282e00-68dc-4215-82ef-d60918af004d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:21.797019 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19282e00-68dc-4215-82ef-d60918af004d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:21.797451 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19282e00-68dc-4215-82ef-d60918af004d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:21.797870 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19282e00-68dc-4215-82ef-d60918af004d None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:21.798800 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19282e00-68dc-4215-82ef-d60918af004d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:21.799251 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19282e00-68dc-4215-82ef-d60918af004d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:21.801431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19282e00-68dc-4215-82ef-d60918af004d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:21.814978 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 476/958] 10.222.0.44 () {56 vars in 1115 bytes} [Thu Jun 23 19:49:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:21.816585 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de8e03-72bd-4037-8696-d9a44112b880 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:21.893296 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:21.920058 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5de8e03-72bd-4037-8696-d9a44112b880 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:21.955903 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc74303-91ee-44ed-800d-c45cc73a7e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:21.957028 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc74303-91ee-44ed-800d-c45cc73a7e67 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:21.957461 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc74303-91ee-44ed-800d-c45cc73a7e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:21.957882 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc74303-91ee-44ed-800d-c45cc73a7e67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:21.968967 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc74303-91ee-44ed-800d-c45cc73a7e67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:21.970756 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc74303-91ee-44ed-800d-c45cc73a7e67 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:21.989678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-2fc74303-91ee-44ed-800d-c45cc73a7e67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:21.989678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:21.989678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:21.989678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:21.989678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:21.989678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:21.989678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:21.989678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:21.989678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:21.989678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:21.989678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:21.989678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:21.989678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:49:21.989678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:49:21.989678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:49:21.990853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:49:21.990853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:21.990853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:21.990853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:21.990853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:21.990853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:21.990853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:21.990853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:21.990853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:21.990853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:21.990853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:21.990853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:21.990853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:21.990853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:21.990853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:21.990853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:21.990853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:21.992440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:49:21.992440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:49:21.992440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:21.992440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:21.998723 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 483/959] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:22.000216 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19282e00-68dc-4215-82ef-d60918af004d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:22.043844 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:22.108612 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19282e00-68dc-4215-82ef-d60918af004d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:22.145323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396b9e5-e51d-4df1-ada8-840d4f8b6c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:22.146262 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396b9e5-e51d-4df1-ada8-840d4f8b6c9e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:22.146702 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396b9e5-e51d-4df1-ada8-840d4f8b6c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:22.147120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3396b9e5-e51d-4df1-ada8-840d4f8b6c9e None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:22.147983 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396b9e5-e51d-4df1-ada8-840d4f8b6c9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:22.148459 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396b9e5-e51d-4df1-ada8-840d4f8b6c9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:22.149986 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3396b9e5-e51d-4df1-ada8-840d4f8b6c9e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:22.167345 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 477/960] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:22.169022 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc74303-91ee-44ed-800d-c45cc73a7e67 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:22.280592 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc74303-91ee-44ed-800d-c45cc73a7e67 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:22.308706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:22.317426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff299c-7efa-4800-9ad1-ee9c4439c395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:22.318248 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff299c-7efa-4800-9ad1-ee9c4439c395 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:22.318786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff299c-7efa-4800-9ad1-ee9c4439c395 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:22.319061 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff299c-7efa-4800-9ad1-ee9c4439c395 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:22.326661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bff299c-7efa-4800-9ad1-ee9c4439c395 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:22.341293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bff299c-7efa-4800-9ad1-ee9c4439c395 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:22.343368 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-5bff299c-7efa-4800-9ad1-ee9c4439c395 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 19:49:22.343368 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:22.343368 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:22.343368 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:22.343368 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:22.343368 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:22.343368 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:22.343368 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:22.343368 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:22.343368 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:22.343368 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:22.343368 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:22.343368 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:49:22.343368 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:49:22.343368 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:49:22.343368 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:49:22.344291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:22.344291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:22.344291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:49:22.344291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:49:22.344291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:49:22.344291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:49:22.344291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:22.344291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:22.344291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:22.344291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:22.344291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:22.344291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:22.344291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:22.344291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:22.344291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:22.344291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:22.344291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:22.344291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:22.345262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:22.345262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:22.345262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:49:22.345262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:49:22.345262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:49:22.345262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:49:22.345262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:49:22.345262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:49:22.345262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 19:49:22.345262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:22.348409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 484/961] 10.222.0.44 () {56 vars in 1117 bytes} [Thu Jun 23 19:49:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:22.351557 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396b9e5-e51d-4df1-ada8-840d4f8b6c9e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:22.464167 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:22.468401 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3396b9e5-e51d-4df1-ada8-840d4f8b6c9e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:22.504671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cc34a6-353c-48cc-ab2c-409338168742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:22.505350 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc34a6-353c-48cc-ab2c-409338168742 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:22.505637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc34a6-353c-48cc-ab2c-409338168742 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:22.505914 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc34a6-353c-48cc-ab2c-409338168742 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:22.506811 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cc34a6-353c-48cc-ab2c-409338168742 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:22.507126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cc34a6-353c-48cc-ab2c-409338168742 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:22.508815 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cc34a6-353c-48cc-ab2c-409338168742 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:22.530987 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 478/962] 10.222.0.44 () {56 vars in 1149 bytes} [Thu Jun 23 19:49:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:22.532349 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff299c-7efa-4800-9ad1-ee9c4439c395 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:22.540366 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b12a4f-a14a-4863-bb47-7ea2a82c4679 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:22.540661 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b12a4f-a14a-4863-bb47-7ea2a82c4679 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:22.540937 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b12a4f-a14a-4863-bb47-7ea2a82c4679 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:22.542278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 479/963] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:49:22.546965 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d552e9-9cad-418c-81dc-756405111415 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:22.547249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d552e9-9cad-418c-81dc-756405111415 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:22.547527 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d552e9-9cad-418c-81dc-756405111415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:22.563701 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b6d552e9-9cad-418c-81dc-756405111415 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:22.564903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b6d552e9-9cad-418c-81dc-756405111415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:22.645044 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 480/964] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 19:49:22.655964 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff299c-7efa-4800-9ad1-ee9c4439c395 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:22.664982 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d552e9-9cad-418c-81dc-756405111415 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:22.701029 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041fd7fe-bef9-4954-865f-05732c0ed068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:22.702639 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041fd7fe-bef9-4954-865f-05732c0ed068 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:22.703024 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041fd7fe-bef9-4954-865f-05732c0ed068 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:22.703423 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041fd7fe-bef9-4954-865f-05732c0ed068 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:22.759360 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:22.772827 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d552e9-9cad-418c-81dc-756405111415 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:22.780654 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:22.799219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041fd7fe-bef9-4954-865f-05732c0ed068 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:22.801448 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041fd7fe-bef9-4954-865f-05732c0ed068 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:22.823047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98ccea-43f6-4eba-b3d1-be5e00428583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:22.824076 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98ccea-43f6-4eba-b3d1-be5e00428583 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:22.824531 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98ccea-43f6-4eba-b3d1-be5e00428583 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:22.824937 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98ccea-43f6-4eba-b3d1-be5e00428583 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:22.825666 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98ccea-43f6-4eba-b3d1-be5e00428583 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:22.827896 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98ccea-43f6-4eba-b3d1-be5e00428583 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:22.883990 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.sql.core [None req-3b98ccea-43f6-4eba-b3d1-be5e00428583 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 19:49:22.883990 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 23 19:49:22.883990 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [parameters: {'user_id': 'bf1318798b0345dfbd2cb62a22f4ed46', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 23 19:49:22.883990 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80363) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 19:49:22.893734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-3b98ccea-43f6-4eba-b3d1-be5e00428583 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 23 19:49:22.893734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:22.893734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 19:49:22.893734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 19:49:22.893734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 19:49:22.893734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 19:49:22.893734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 19:49:22.893734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 19:49:22.893734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 19:49:22.893734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application conn.query(q) Jun 23 19:49:22.893734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 19:49:22.893734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 19:49:22.893734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 19:49:22.893734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result.read() Jun 23 19:49:22.893734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 19:49:22.895142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 19:49:22.895142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 19:49:22.895142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 19:49:22.895142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 19:49:22.895142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 19:49:22.895142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 19:49:22.895142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 19:49:22.895142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 19:49:22.895142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:22.895142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 19:49:22.895142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:22.895142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:22.895142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 19:49:22.895142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 19:49:22.895142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 23 19:49:22.895142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 23 19:49:22.895142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 19:49:22.895142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application next(self.gen) Jun 23 19:49:22.896109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 19:49:22.896109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application yield resource Jun 23 19:49:22.896109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 19:49:22.896109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application next(self.gen) Jun 23 19:49:22.896109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 19:49:22.896109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 19:49:22.896109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 19:49:22.896109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 19:49:22.896109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 19:49:22.896109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise self.value Jun 23 19:49:22.896109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 19:49:22.896109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 19:49:22.896109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 19:49:22.896109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application session.commit() Jun 23 19:49:22.896109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 19:49:22.896109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 19:49:22.896109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 19:49:22.897130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 19:49:22.897130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 19:49:22.897130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.session.flush() Jun 23 19:49:22.897130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 19:49:22.897130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 19:49:22.897130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 19:49:22.897130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 19:49:22.897130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 19:49:22.897130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application compat.raise_( Jun 23 19:49:22.897130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 19:49:22.897130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception Jun 23 19:49:22.897130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 19:49:22.897130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 19:49:22.897130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 19:49:22.897130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 19:49:22.897130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 19:49:22.897130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 19:49:22.898062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 19:49:22.898062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 19:49:22.898062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 23 19:49:22.898062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 23 19:49:22.898062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 19:49:22.898062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 19:49:22.898062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 19:49:22.898062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 19:49:22.898062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 19:49:22.898062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 19:49:22.898062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 19:49:22.898062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 19:49:22.898062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 19:49:22.898062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 19:49:22.898062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 19:49:22.898062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception Jun 23 19:49:22.899003 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 19:49:22.899003 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 19:49:22.899003 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 19:49:22.899003 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 19:49:22.899003 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 19:49:22.899003 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 19:49:22.899003 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 19:49:22.899003 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application conn.query(q) Jun 23 19:49:22.899003 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 19:49:22.899003 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 19:49:22.899003 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 19:49:22.899003 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result.read() Jun 23 19:49:22.899003 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 19:49:22.899003 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 19:49:22.899003 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 19:49:22.899003 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 19:49:22.899873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 19:49:22.899873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 19:49:22.899873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 19:49:22.899873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 19:49:22.899873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 19:49:22.899873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 23 19:49:22.899873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [parameters: {'user_id': 'bf1318798b0345dfbd2cb62a22f4ed46', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 23 19:49:22.899873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 19:49:22.899873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:22.899873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:49:22.899873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:22.899873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:22.899873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:22.899873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:22.899873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:22.908747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:22.908747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:22.908747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:22.908747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:22.908747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:22.908747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:22.908747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:22.908747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 23 19:49:22.908747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 23 19:49:22.908747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:22.908747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:22.908747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:49:22.908747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:49:22.908747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:49:22.908747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:49:22.908747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 23 19:49:22.908747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 23 19:49:22.909926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 23 19:49:22.909926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 23 19:49:22.909926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 19:49:22.909926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 19:49:22.909926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 23 19:49:22.909926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:22.909926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 481/965] 10.222.0.44 () {60 vars in 1131 bytes} [Thu Jun 23 19:49:22 2022] POST /identity/v3/users => generated 160 bytes in 232 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:22.909926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98ccea-43f6-4eba-b3d1-be5e00428583 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:22.913416 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 485/966] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:49:22 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 19:49:22.923894 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041fd7fe-bef9-4954-865f-05732c0ed068 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:23.030889 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98ccea-43f6-4eba-b3d1-be5e00428583 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:23.031752 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:23.051277 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041fd7fe-bef9-4954-865f-05732c0ed068 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:23.067640 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2426aa0-f0c4-44a5-b27f-5eddf3642af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:23.068633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2426aa0-f0c4-44a5-b27f-5eddf3642af4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:23.069013 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2426aa0-f0c4-44a5-b27f-5eddf3642af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:23.069383 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2426aa0-f0c4-44a5-b27f-5eddf3642af4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:23.077420 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2426aa0-f0c4-44a5-b27f-5eddf3642af4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:23.079186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2426aa0-f0c4-44a5-b27f-5eddf3642af4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:23.093135 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd496bd-2c66-4661-991b-a13556abe688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:23.093923 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd496bd-2c66-4661-991b-a13556abe688 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:23.094201 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd496bd-2c66-4661-991b-a13556abe688 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:23.094471 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd496bd-2c66-4661-991b-a13556abe688 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:23.094822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-f2426aa0-f0c4-44a5-b27f-5eddf3642af4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 19:49:23.094822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:23.094822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:23.094822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:23.094822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:23.094822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:23.094822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:23.094822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:23.094822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:23.094822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:23.094822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:23.094822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:23.094822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:49:23.094822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:49:23.094822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:49:23.094822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:49:23.096521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:23.096521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:23.096521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:49:23.096521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:49:23.096521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:49:23.096521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:49:23.096521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:23.096521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:23.096521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:23.096521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:23.096521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:23.096521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:23.096521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:23.096521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:23.096521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:23.096521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:23.096521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:23.096521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:23.099334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:23.099334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:23.099334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:49:23.099334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:49:23.099334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:49:23.099334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:49:23.099334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:49:23.099334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:49:23.099334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 19:49:23.099334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:23.108318 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 482/967] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:49:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:23.108318 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2426aa0-f0c4-44a5-b27f-5eddf3642af4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:23.108700 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd496bd-2c66-4661-991b-a13556abe688 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:23.109482 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd496bd-2c66-4661-991b-a13556abe688 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:23.114823 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:23.132086 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-7cd496bd-2c66-4661-991b-a13556abe688 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:23.132086 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:23.132086 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:23.132086 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:23.132086 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:23.132086 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:23.132086 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:23.132086 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:23.132086 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:23.132086 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:23.132086 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:23.132086 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:23.132086 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:49:23.132086 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:49:23.132086 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:49:23.133910 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:49:23.133910 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:23.133910 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:23.133910 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:23.133910 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:23.133910 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:23.133910 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:23.133910 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:23.133910 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:23.133910 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:23.133910 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:23.133910 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:23.133910 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:23.133910 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:23.133910 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:23.133910 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:23.133910 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:23.135131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:49:23.135131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:49:23.135131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:23.135131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:23.138807 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 486/968] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:23.140231 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd496bd-2c66-4661-991b-a13556abe688 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:23.219762 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2426aa0-f0c4-44a5-b27f-5eddf3642af4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:23.265319 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd496bd-2c66-4661-991b-a13556abe688 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:23.270291 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56480d6-d4b6-4b89-b232-b8a96ba06a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:23.271235 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56480d6-d4b6-4b89-b232-b8a96ba06a97 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:23.271740 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56480d6-d4b6-4b89-b232-b8a96ba06a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:23.272213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56480d6-d4b6-4b89-b232-b8a96ba06a97 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:23.273256 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56480d6-d4b6-4b89-b232-b8a96ba06a97 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:23.274275 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56480d6-d4b6-4b89-b232-b8a96ba06a97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:23.276460 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56480d6-d4b6-4b89-b232-b8a96ba06a97 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:23.302391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335272f-754d-47d8-8a8d-79105f4c459b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:23.303046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335272f-754d-47d8-8a8d-79105f4c459b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:23.303322 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335272f-754d-47d8-8a8d-79105f4c459b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:23.303590 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335272f-754d-47d8-8a8d-79105f4c459b None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:23.304391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335272f-754d-47d8-8a8d-79105f4c459b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:23.304695 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335272f-754d-47d8-8a8d-79105f4c459b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:23.306178 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335272f-754d-47d8-8a8d-79105f4c459b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:23.313796 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:23.324127 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 487/969] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:49:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:23.326450 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335272f-754d-47d8-8a8d-79105f4c459b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:23.336567 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 483/970] 10.222.0.44 () {56 vars in 1101 bytes} [Thu Jun 23 19:49:23 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:23.380773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d382367-ea48-46f8-a6bc-09f723d027b6 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:23.381058 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d382367-ea48-46f8-a6bc-09f723d027b6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:23.381393 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d382367-ea48-46f8-a6bc-09f723d027b6 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:23.382987 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 484/971] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:49:23.388819 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96836708-d6ea-43b3-abc9-fe9e2ef25742 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:23.389240 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96836708-d6ea-43b3-abc9-fe9e2ef25742 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:23.389378 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96836708-d6ea-43b3-abc9-fe9e2ef25742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:23.408945 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-96836708-d6ea-43b3-abc9-fe9e2ef25742 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:23.410633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-96836708-d6ea-43b3-abc9-fe9e2ef25742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:23.464245 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335272f-754d-47d8-8a8d-79105f4c459b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:23.480023 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:23.491227 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 485/972] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 19:49:23.515695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96836708-d6ea-43b3-abc9-fe9e2ef25742 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:23.516636 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3eca4a-febc-4dc8-b2d7-8b7a21060e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:23.517801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3eca4a-febc-4dc8-b2d7-8b7a21060e6b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:23.518240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3eca4a-febc-4dc8-b2d7-8b7a21060e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:23.518516 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3eca4a-febc-4dc8-b2d7-8b7a21060e6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:23.527862 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3eca4a-febc-4dc8-b2d7-8b7a21060e6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:23.529672 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3eca4a-febc-4dc8-b2d7-8b7a21060e6b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:23.544273 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-ee3eca4a-febc-4dc8-b2d7-8b7a21060e6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:49:23.544273 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:23.544273 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:23.544273 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:23.544273 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:23.544273 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:23.544273 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:23.544273 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:23.544273 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:23.544273 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:23.544273 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:23.544273 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:23.544273 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:49:23.544273 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:49:23.544273 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:49:23.545404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:49:23.545404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:23.545404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:23.545404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:23.545404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:23.545404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:23.545404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:23.545404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:23.545404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:23.545404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:23.545404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:23.545404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:23.545404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:23.545404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:23.545404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:23.545404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:23.545404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:23.546457 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:49:23.546457 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:49:23.546457 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:49:23.546457 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:49:23.546457 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:49:23.546457 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:49:23.546457 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:49:23.546457 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:23.546457 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 488/973] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:49:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:49:23.552152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3eca4a-febc-4dc8-b2d7-8b7a21060e6b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:23.622393 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:23.637691 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96836708-d6ea-43b3-abc9-fe9e2ef25742 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:23.668946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3eca4a-febc-4dc8-b2d7-8b7a21060e6b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:23.677456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242c0d9a-0273-42b8-897a-662f0f434c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:23.678259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c0d9a-0273-42b8-897a-662f0f434c36 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:23.678536 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c0d9a-0273-42b8-897a-662f0f434c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:23.678810 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c0d9a-0273-42b8-897a-662f0f434c36 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:23.679454 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242c0d9a-0273-42b8-897a-662f0f434c36 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:23.680861 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242c0d9a-0273-42b8-897a-662f0f434c36 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:23.693545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-242c0d9a-0273-42b8-897a-662f0f434c36 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 19:49:23.693545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:23.693545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 19:49:23.693545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 19:49:23.693545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 19:49:23.693545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 19:49:23.693545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 19:49:23.693545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:23.693545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 19:49:23.693545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:23.693545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:49:23.693545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:23.693545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:23.693545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:23.693545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:23.693545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:23.693545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:23.694579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:23.694579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:23.694579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:23.694579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:23.694579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:23.694579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:23.694579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 19:49:23.694579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 19:49:23.694579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 19:49:23.694579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 19:49:23.694579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:23.694579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:23.694579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:23.694579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:23.694579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:23.694579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:23.696051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:23.696051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:23.696051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:23.696051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:23.696051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:23.696051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:23.696051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:23.696051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:23.696051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:23.696051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:23.696051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 19:49:23.696051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:23.696051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 19:49:23.696051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:23.696051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 486/974] 10.222.0.44 () {56 vars in 1126 bytes} [Thu Jun 23 19:49:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:49:23.700906 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242c0d9a-0273-42b8-897a-662f0f434c36 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:23.708087 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f9ee3-2cd2-4e1d-ba55-48b1e1efc869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:23.708772 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f9ee3-2cd2-4e1d-ba55-48b1e1efc869 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:23.709048 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f9ee3-2cd2-4e1d-ba55-48b1e1efc869 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:23.709319 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f9ee3-2cd2-4e1d-ba55-48b1e1efc869 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:23.710098 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f9ee3-2cd2-4e1d-ba55-48b1e1efc869 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:23.710402 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f9ee3-2cd2-4e1d-ba55-48b1e1efc869 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:23.711837 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f9ee3-2cd2-4e1d-ba55-48b1e1efc869 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:23.727901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 489/975] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:49:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:23.729893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f9ee3-2cd2-4e1d-ba55-48b1e1efc869 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:23.773814 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:23.813760 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242c0d9a-0273-42b8-897a-662f0f434c36 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:23.831032 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f9ee3-2cd2-4e1d-ba55-48b1e1efc869 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:23.846066 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af4936f-b9b5-49ec-bcfd-deddc5ea2117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:23.846739 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4936f-b9b5-49ec-bcfd-deddc5ea2117 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:23.847157 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4936f-b9b5-49ec-bcfd-deddc5ea2117 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:23.847455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:23.847553 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4936f-b9b5-49ec-bcfd-deddc5ea2117 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:23.848259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af4936f-b9b5-49ec-bcfd-deddc5ea2117 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:23.848987 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af4936f-b9b5-49ec-bcfd-deddc5ea2117 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:23.850372 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af4936f-b9b5-49ec-bcfd-deddc5ea2117 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:23.855120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 487/976] 10.222.0.44 () {56 vars in 1122 bytes} [Thu Jun 23 19:49:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:23.860933 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5f656-037d-4d1e-b657-31ea5711a1f4 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:23.861226 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5f656-037d-4d1e-b657-31ea5711a1f4 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:23.861505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5f656-037d-4d1e-b657-31ea5711a1f4 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:23.862752 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 488/977] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:23.863944 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5f656-037d-4d1e-b657-31ea5711a1f4 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:23.864882 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c941d-0c8e-41c2-9ba1-58adb8b17cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:23.865528 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c941d-0c8e-41c2-9ba1-58adb8b17cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:23.865810 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c941d-0c8e-41c2-9ba1-58adb8b17cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:23.866082 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c941d-0c8e-41c2-9ba1-58adb8b17cd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:23.866907 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329c941d-0c8e-41c2-9ba1-58adb8b17cd4 None admin] RBAC: Adding query filter params (role.id=ec091d6061664ed697565c76e216b5b4, scope.project.id=155180ba63c446ca889a6eb3f94cc31a, user.id=1d37f369749143ce813c693adc5f2ec7) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:23.867293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329c941d-0c8e-41c2-9ba1-58adb8b17cd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:23.868772 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329c941d-0c8e-41c2-9ba1-58adb8b17cd4 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:23.874619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 490/978] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:49:23 2022] GET /identity/v3/role_assignments?user.id=1d37f369749143ce813c693adc5f2ec7&scope.project.id=155180ba63c446ca889a6eb3f94cc31a&role.id=ec091d6061664ed697565c76e216b5b4 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:23.875900 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf83a2-493b-4f6f-b4bb-0a10a1c62251 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:23.876235 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf83a2-493b-4f6f-b4bb-0a10a1c62251 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:23.876542 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf83a2-493b-4f6f-b4bb-0a10a1c62251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:23.912913 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3caf83a2-493b-4f6f-b4bb-0a10a1c62251 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:23.914144 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3caf83a2-493b-4f6f-b4bb-0a10a1c62251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:23.980720 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5f656-037d-4d1e-b657-31ea5711a1f4 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:23.992730 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 491/979] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:24.013063 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:24.016036 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf83a2-493b-4f6f-b4bb-0a10a1c62251 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:24.027041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace67cf6-5063-423b-b1e7-3678a954d779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:24.031767 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace67cf6-5063-423b-b1e7-3678a954d779 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:24.031767 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace67cf6-5063-423b-b1e7-3678a954d779 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:24.031767 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace67cf6-5063-423b-b1e7-3678a954d779 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:24.031767 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace67cf6-5063-423b-b1e7-3678a954d779 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:24.031767 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace67cf6-5063-423b-b1e7-3678a954d779 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:24.031767 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace67cf6-5063-423b-b1e7-3678a954d779 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:24.041864 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 489/980] 10.222.0.44 () {56 vars in 1122 bytes} [Thu Jun 23 19:49:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:24.048050 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace67cf6-5063-423b-b1e7-3678a954d779 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:24.130055 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf83a2-493b-4f6f-b4bb-0a10a1c62251 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:24.144375 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:24.161153 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace67cf6-5063-423b-b1e7-3678a954d779 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:24.168330 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1c795-8a16-43e4-baaa-f32b1bba413f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:24.168995 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1c795-8a16-43e4-baaa-f32b1bba413f None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:24.169279 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1c795-8a16-43e4-baaa-f32b1bba413f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:24.169552 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1c795-8a16-43e4-baaa-f32b1bba413f None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:24.170323 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1c795-8a16-43e4-baaa-f32b1bba413f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:24.171874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1c795-8a16-43e4-baaa-f32b1bba413f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:24.197219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c471b4-7fa4-4d94-b3f1-c3f711ed6342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:24.197951 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c471b4-7fa4-4d94-b3f1-c3f711ed6342 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:24.198244 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c471b4-7fa4-4d94-b3f1-c3f711ed6342 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:24.198523 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c471b4-7fa4-4d94-b3f1-c3f711ed6342 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:24.199304 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c471b4-7fa4-4d94-b3f1-c3f711ed6342 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:24.200707 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c471b4-7fa4-4d94-b3f1-c3f711ed6342 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:24.273569 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 492/981] 10.222.0.44 () {60 vars in 1141 bytes} [Thu Jun 23 19:49:24 2022] POST /identity/v3/services => generated 242 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 19:49:24.284996 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 490/982] 10.222.0.44 () {60 vars in 1144 bytes} [Thu Jun 23 19:49:24 2022] POST /identity/v3/endpoints => generated 349 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:49:25.954265 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ff2cd-2aa1-4b15-a31e-56aa51d7b774 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:25.954793 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ff2cd-2aa1-4b15-a31e-56aa51d7b774 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:25.954793 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ff2cd-2aa1-4b15-a31e-56aa51d7b774 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:25.956341 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 493/983] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:25.961985 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ae1f4-57d3-4c81-901f-acdb8dd23c96 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:25.962413 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ae1f4-57d3-4c81-901f-acdb8dd23c96 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:25.962610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ae1f4-57d3-4c81-901f-acdb8dd23c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:25.979397 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-075ae1f4-57d3-4c81-901f-acdb8dd23c96 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:25.980596 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-075ae1f4-57d3-4c81-901f-acdb8dd23c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:26.021657 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:26.053635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 491/984] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:25 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:26.075879 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ff2cd-2aa1-4b15-a31e-56aa51d7b774 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:26.175144 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ff2cd-2aa1-4b15-a31e-56aa51d7b774 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:26.208496 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa27d3-a50e-4aeb-8995-ba4cfd6312e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:26.209202 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa27d3-a50e-4aeb-8995-ba4cfd6312e9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:26.209482 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa27d3-a50e-4aeb-8995-ba4cfd6312e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:26.209752 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa27d3-a50e-4aeb-8995-ba4cfd6312e9 None admin] PATH_INFO: `/v3/services/image` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:26.210395 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa27d3-a50e-4aeb-8995-ba4cfd6312e9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:26.211777 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa27d3-a50e-4aeb-8995-ba4cfd6312e9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:26.224125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-f7fa27d3-a50e-4aeb-8995-ba4cfd6312e9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 19:49:26.224125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:26.224125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 19:49:26.224125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 19:49:26.224125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 19:49:26.224125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 19:49:26.224125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 19:49:26.224125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:26.224125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 19:49:26.224125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:26.224125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:49:26.224125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:26.224125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:26.224125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:26.224125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:26.224125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:26.224125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:26.225213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:26.225213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:26.225213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:26.225213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:26.225213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:26.225213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:26.225213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 19:49:26.225213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 19:49:26.225213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 19:49:26.225213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 19:49:26.225213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:26.225213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:26.225213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:26.225213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:26.225213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:26.225213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:26.226097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:26.226097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:26.226097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:26.226097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:26.226097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:26.226097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:26.226097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:26.226097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:26.226097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:26.226097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:26.226097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 19:49:26.226097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:26.226097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 19:49:26.226097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:26.234653 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 494/985] 10.222.0.44 () {56 vars in 1105 bytes} [Thu Jun 23 19:49:26 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:26.234825 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ae1f4-57d3-4c81-901f-acdb8dd23c96 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:26.335785 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ae1f4-57d3-4c81-901f-acdb8dd23c96 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:26.350869 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7171a0-a702-405a-977a-51e28cbe5681 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:26.351253 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7171a0-a702-405a-977a-51e28cbe5681 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:26.351534 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7171a0-a702-405a-977a-51e28cbe5681 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:26.352945 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 495/986] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:26.357983 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54036190-d41e-4612-82db-91dbf611728d None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:26.358358 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54036190-d41e-4612-82db-91dbf611728d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:26.358694 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54036190-d41e-4612-82db-91dbf611728d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:26.375460 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ecd945-2a0f-4141-8236-300b47e98314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:26.375814 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-54036190-d41e-4612-82db-91dbf611728d None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:26.376869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecd945-2a0f-4141-8236-300b47e98314 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:26.377032 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-54036190-d41e-4612-82db-91dbf611728d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:26.377483 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecd945-2a0f-4141-8236-300b47e98314 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:26.377937 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecd945-2a0f-4141-8236-300b47e98314 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:26.379155 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ecd945-2a0f-4141-8236-300b47e98314 None admin] RBAC: Adding query filter params (name=image) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:26.379644 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ecd945-2a0f-4141-8236-300b47e98314 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:26.381810 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ecd945-2a0f-4141-8236-300b47e98314 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:26.387142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 492/987] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:26.392483 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ecd945-2a0f-4141-8236-300b47e98314 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:26.407555 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:26.428151 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:26.452551 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 496/988] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:26 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 19:49:26.453722 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b95de-0f4f-4f53-881e-e6511c6d5d84 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:26.454013 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b95de-0f4f-4f53-881e-e6511c6d5d84 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:26.454290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b95de-0f4f-4f53-881e-e6511c6d5d84 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:26.456275 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 497/989] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:26.461477 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1117cb-91ab-4635-b68b-0b2c79bce2b6 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:26.461786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1117cb-91ab-4635-b68b-0b2c79bce2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:26.461986 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1117cb-91ab-4635-b68b-0b2c79bce2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:26.479273 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-7e1117cb-91ab-4635-b68b-0b2c79bce2b6 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:26.485839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7e1117cb-91ab-4635-b68b-0b2c79bce2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:26.497106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ecd945-2a0f-4141-8236-300b47e98314 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:26.530685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f38842-3717-4df9-bdb5-ccaee2d6828e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:26.531348 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f38842-3717-4df9-bdb5-ccaee2d6828e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:26.531625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f38842-3717-4df9-bdb5-ccaee2d6828e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:26.531895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f38842-3717-4df9-bdb5-ccaee2d6828e None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:26.532624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f38842-3717-4df9-bdb5-ccaee2d6828e None admin] RBAC: Adding query filter params (type=image) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:26.532926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f38842-3717-4df9-bdb5-ccaee2d6828e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:26.534369 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f38842-3717-4df9-bdb5-ccaee2d6828e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:26.539615 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 493/990] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:26 2022] GET /identity/v3/services?type=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:26.540507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f38842-3717-4df9-bdb5-ccaee2d6828e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:26.553512 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 498/991] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:26 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 19:49:26.578089 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1117cb-91ab-4635-b68b-0b2c79bce2b6 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:26.667149 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:26.672049 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f38842-3717-4df9-bdb5-ccaee2d6828e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:26.684338 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:26.690361 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1117cb-91ab-4635-b68b-0b2c79bce2b6 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:26.715248 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee8362-12ac-4e31-be1b-1c14250496aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:26.715248 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee8362-12ac-4e31-be1b-1c14250496aa None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:26.715248 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee8362-12ac-4e31-be1b-1c14250496aa None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:26.715248 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee8362-12ac-4e31-be1b-1c14250496aa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:26.715635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee8362-12ac-4e31-be1b-1c14250496aa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:26.715635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee8362-12ac-4e31-be1b-1c14250496aa None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:26.728872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-fdee8362-12ac-4e31-be1b-1c14250496aa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 19:49:26.728872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:26.728872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 19:49:26.728872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 19:49:26.728872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 19:49:26.728872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 19:49:26.728872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 19:49:26.728872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:26.728872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 19:49:26.728872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:26.728872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:49:26.728872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:26.728872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:26.728872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:26.728872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:26.728872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:26.728872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:26.729894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:26.729894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:26.729894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:26.729894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:26.729894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:26.729894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:26.729894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 19:49:26.729894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 19:49:26.729894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 19:49:26.729894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 19:49:26.729894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:26.729894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:26.729894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:26.729894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:26.729894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:26.729894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:26.730746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:26.730746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:26.730746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:26.730746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:26.730746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:26.730746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:26.730746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:26.730746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:26.730746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:26.730746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:26.730746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 19:49:26.730746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:26.730746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 19:49:26.730746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:26.732342 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f5151-42e3-4bf4-91a8-506d4889c40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:26.733261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f5151-42e3-4bf4-91a8-506d4889c40d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:26.733570 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f5151-42e3-4bf4-91a8-506d4889c40d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:26.733711 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 494/992] 10.222.0.44 () {56 vars in 1126 bytes} [Thu Jun 23 19:49:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:26.733878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f5151-42e3-4bf4-91a8-506d4889c40d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:26.734660 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95f5151-42e3-4bf4-91a8-506d4889c40d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:26.736072 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95f5151-42e3-4bf4-91a8-506d4889c40d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:26.738996 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee8362-12ac-4e31-be1b-1c14250496aa None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:26.748775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-d95f5151-42e3-4bf4-91a8-506d4889c40d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 19:49:26.748775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:26.748775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 19:49:26.748775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 19:49:26.748775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 19:49:26.748775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 19:49:26.748775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 19:49:26.748775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:26.748775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 19:49:26.748775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:26.748775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:49:26.748775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:26.748775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:26.748775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:26.748775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:26.748775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:26.748775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:26.749891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:26.749891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:26.749891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:26.749891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:26.749891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:26.749891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:26.749891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 19:49:26.749891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 19:49:26.749891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 19:49:26.749891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 19:49:26.749891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:26.749891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:26.749891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:26.749891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:26.749891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:26.749891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:26.750723 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:26.750723 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:26.750723 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:26.750723 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:26.750723 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:26.750723 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:26.750723 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:26.750723 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:26.750723 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:26.750723 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:26.750723 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 19:49:26.750723 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:26.750723 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 19:49:26.750723 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:26.750723 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 499/993] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:49:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:49:26.755630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f5151-42e3-4bf4-91a8-506d4889c40d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:26.850768 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee8362-12ac-4e31-be1b-1c14250496aa None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:26.862059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f5151-42e3-4bf4-91a8-506d4889c40d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:26.886735 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608e42a-f8f8-4d31-a6a4-b375cab029a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:26.887402 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608e42a-f8f8-4d31-a6a4-b375cab029a2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:26.887682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608e42a-f8f8-4d31-a6a4-b375cab029a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:26.887956 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9608e42a-f8f8-4d31-a6a4-b375cab029a2 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:26.888681 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608e42a-f8f8-4d31-a6a4-b375cab029a2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:26.888984 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608e42a-f8f8-4d31-a6a4-b375cab029a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:26.890340 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9608e42a-f8f8-4d31-a6a4-b375cab029a2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:26.895935 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 495/994] 10.222.0.44 () {56 vars in 1122 bytes} [Thu Jun 23 19:49:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:26.901443 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608e42a-f8f8-4d31-a6a4-b375cab029a2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:26.902300 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640865c3-5f1e-450c-bcc8-c9818689655d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:26.902959 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640865c3-5f1e-450c-bcc8-c9818689655d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:26.903239 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640865c3-5f1e-450c-bcc8-c9818689655d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:26.903511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640865c3-5f1e-450c-bcc8-c9818689655d None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:26.904217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640865c3-5f1e-450c-bcc8-c9818689655d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:26.904549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640865c3-5f1e-450c-bcc8-c9818689655d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:26.905919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640865c3-5f1e-450c-bcc8-c9818689655d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:26.911447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 500/995] 10.222.0.44 () {56 vars in 1112 bytes} [Thu Jun 23 19:49:26 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:26.917002 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640865c3-5f1e-450c-bcc8-c9818689655d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:26.934776 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:26.945743 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:27.012480 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9608e42a-f8f8-4d31-a6a4-b375cab029a2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:27.026785 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640865c3-5f1e-450c-bcc8-c9818689655d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:27.049125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c22587-375c-4c90-bd54-d4b99cad3df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:27.049872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c22587-375c-4c90-bd54-d4b99cad3df2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:27.050221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c22587-375c-4c90-bd54-d4b99cad3df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:27.050627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c22587-375c-4c90-bd54-d4b99cad3df2 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:27.051328 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c22587-375c-4c90-bd54-d4b99cad3df2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:27.051632 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c22587-375c-4c90-bd54-d4b99cad3df2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:27.053029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c22587-375c-4c90-bd54-d4b99cad3df2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:27.058335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 496/996] 10.222.0.44 () {56 vars in 1122 bytes} [Thu Jun 23 19:49:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:27.063866 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c22587-375c-4c90-bd54-d4b99cad3df2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:27.064791 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe48580e-e637-4a2a-a04f-30a794269184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:27.065452 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48580e-e637-4a2a-a04f-30a794269184 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:27.065731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48580e-e637-4a2a-a04f-30a794269184 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:27.066518 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48580e-e637-4a2a-a04f-30a794269184 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:27.067226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe48580e-e637-4a2a-a04f-30a794269184 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:27.067566 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe48580e-e637-4a2a-a04f-30a794269184 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:27.068977 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe48580e-e637-4a2a-a04f-30a794269184 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:27.078766 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 501/997] 10.222.0.44 () {56 vars in 1112 bytes} [Thu Jun 23 19:49:26 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:27.079728 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe48580e-e637-4a2a-a04f-30a794269184 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:27.164056 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:27.170072 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:27.174709 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c22587-375c-4c90-bd54-d4b99cad3df2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:27.183194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe48580e-e637-4a2a-a04f-30a794269184 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:27.214779 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a69d6-935a-4db2-ac9c-53cd896664da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:27.215438 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a69d6-935a-4db2-ac9c-53cd896664da None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:27.215728 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a69d6-935a-4db2-ac9c-53cd896664da None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:27.216005 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a69d6-935a-4db2-ac9c-53cd896664da None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:27.216844 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a69d6-935a-4db2-ac9c-53cd896664da None admin] RBAC: Adding query filter params (interface=admin, service_id=21cbf623842f4ea5bd11af6c3805be3f, region_id=RegionOne) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:27.217186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a69d6-935a-4db2-ac9c-53cd896664da None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:27.218338 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f77315-e315-4fa5-9694-658bb630ea46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:27.218561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a69d6-935a-4db2-ac9c-53cd896664da None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:27.219359 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f77315-e315-4fa5-9694-658bb630ea46 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:27.219788 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f77315-e315-4fa5-9694-658bb630ea46 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:27.220221 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f77315-e315-4fa5-9694-658bb630ea46 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:27.221180 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f77315-e315-4fa5-9694-658bb630ea46 None admin] RBAC: Adding query filter params (interface=public, service_id=a0b2363d6d224f279b47a91720a32419, region_id=RegionOne) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:27.221495 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f77315-e315-4fa5-9694-658bb630ea46 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:27.223474 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f77315-e315-4fa5-9694-658bb630ea46 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:27.229380 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 497/998] 10.222.0.44 () {56 vars in 1249 bytes} [Thu Jun 23 19:49:27 2022] GET /identity/v3/endpoints?service_id=21cbf623842f4ea5bd11af6c3805be3f&interface=admin®ion_id=RegionOne => generated 193 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:27.230261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a69d6-935a-4db2-ac9c-53cd896664da None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:27.236506 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 502/999] 10.222.0.44 () {56 vars in 1251 bytes} [Thu Jun 23 19:49:27 2022] GET /identity/v3/endpoints?service_id=a0b2363d6d224f279b47a91720a32419&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:27.241788 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f77315-e315-4fa5-9694-658bb630ea46 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:27.340278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a69d6-935a-4db2-ac9c-53cd896664da None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:27.348597 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f77315-e315-4fa5-9694-658bb630ea46 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:27.375384 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7d084-f238-40ab-8907-8fbab675d97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:27.376048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7d084-f238-40ab-8907-8fbab675d97d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:27.376464 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7d084-f238-40ab-8907-8fbab675d97d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:27.376657 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7d084-f238-40ab-8907-8fbab675d97d None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:27.377313 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7d084-f238-40ab-8907-8fbab675d97d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:27.378744 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7d084-f238-40ab-8907-8fbab675d97d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:27.396332 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 498/1000] 10.222.0.44 () {56 vars in 1087 bytes} [Thu Jun 23 19:49:27 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 19:49:27.398246 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227ef7c3-4fae-4904-a3d7-29ef4bf072ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:27.399132 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ef7c3-4fae-4904-a3d7-29ef4bf072ed None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:27.399489 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ef7c3-4fae-4904-a3d7-29ef4bf072ed None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:27.399837 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ef7c3-4fae-4904-a3d7-29ef4bf072ed None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:27.400660 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227ef7c3-4fae-4904-a3d7-29ef4bf072ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:27.402223 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227ef7c3-4fae-4904-a3d7-29ef4bf072ed None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:27.409599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 503/1001] 10.222.0.44 () {56 vars in 1087 bytes} [Thu Jun 23 19:49:27 2022] GET /identity/v3/services => generated 1727 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 19:49:28.631575 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a82dac-c233-4910-9eac-bcf995042466 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:28.632157 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a82dac-c233-4910-9eac-bcf995042466 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:28.632157 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a82dac-c233-4910-9eac-bcf995042466 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:28.633661 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 499/1002] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:28.638583 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85560e-19d6-4dbf-8b99-bd2443bb0268 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:28.638985 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85560e-19d6-4dbf-8b99-bd2443bb0268 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:28.639150 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85560e-19d6-4dbf-8b99-bd2443bb0268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:28.655585 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-cb85560e-19d6-4dbf-8b99-bd2443bb0268 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:28.656788 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-cb85560e-19d6-4dbf-8b99-bd2443bb0268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:28.668935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:28.724981 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 504/1003] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:28 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:28.748195 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a82dac-c233-4910-9eac-bcf995042466 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:28.766743 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:28.846590 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a82dac-c233-4910-9eac-bcf995042466 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:28.879520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88f43c-0b73-4266-a8f5-5f1d09bf42e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:28.880168 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88f43c-0b73-4266-a8f5-5f1d09bf42e3 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:28.880489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88f43c-0b73-4266-a8f5-5f1d09bf42e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:28.880761 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88f43c-0b73-4266-a8f5-5f1d09bf42e3 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:28.881473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb88f43c-0b73-4266-a8f5-5f1d09bf42e3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:28.882883 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb88f43c-0b73-4266-a8f5-5f1d09bf42e3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:28.992345 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 500/1004] 10.222.0.44 () {60 vars in 1141 bytes} [Thu Jun 23 19:49:28 2022] POST /identity/v3/services => generated 242 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:49:29.493392 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853895b7-0fa4-432e-a7f0-f39fa4629e6b None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:29.493588 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853895b7-0fa4-432e-a7f0-f39fa4629e6b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:29.493863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853895b7-0fa4-432e-a7f0-f39fa4629e6b None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:29.495274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 505/1005] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:29.500578 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322e667-fd7e-48d4-9ac5-d640f30925c3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:29.500993 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322e667-fd7e-48d4-9ac5-d640f30925c3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:29.501339 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322e667-fd7e-48d4-9ac5-d640f30925c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:29.518758 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9322e667-fd7e-48d4-9ac5-d640f30925c3 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:29.520908 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9322e667-fd7e-48d4-9ac5-d640f30925c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:29.559394 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e276a4c-b051-46bd-ab3a-91faa290039c None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:29.559693 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e276a4c-b051-46bd-ab3a-91faa290039c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:29.559960 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e276a4c-b051-46bd-ab3a-91faa290039c None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:29.561527 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 506/1006] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:29.566868 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a69b86-b8c8-4cbf-a804-0df79a8ed165 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:29.567161 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a69b86-b8c8-4cbf-a804-0df79a8ed165 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:29.567443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a69b86-b8c8-4cbf-a804-0df79a8ed165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:29.583926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-26a69b86-b8c8-4cbf-a804-0df79a8ed165 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:29.585164 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-26a69b86-b8c8-4cbf-a804-0df79a8ed165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:29.600563 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 501/1007] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:29 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:29.619511 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322e667-fd7e-48d4-9ac5-d640f30925c3 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:29.622956 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:29.658456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 507/1008] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:29 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 19:49:29.682257 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a69b86-b8c8-4cbf-a804-0df79a8ed165 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:29.737090 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322e667-fd7e-48d4-9ac5-d640f30925c3 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:29.770956 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:29.780809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e304835a-ff63-499b-9fc7-0bfe51502220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:29.781613 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304835a-ff63-499b-9fc7-0bfe51502220 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:29.781893 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304835a-ff63-499b-9fc7-0bfe51502220 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:29.782162 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304835a-ff63-499b-9fc7-0bfe51502220 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:29.782809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e304835a-ff63-499b-9fc7-0bfe51502220 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:29.784169 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e304835a-ff63-499b-9fc7-0bfe51502220 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:29.794172 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a69b86-b8c8-4cbf-a804-0df79a8ed165 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:29.797129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-e304835a-ff63-499b-9fc7-0bfe51502220 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 19:49:29.797129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:29.797129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 19:49:29.797129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 19:49:29.797129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 19:49:29.797129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 19:49:29.797129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 19:49:29.797129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:29.797129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 19:49:29.797129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:29.797129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:49:29.797129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:29.797129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:29.797129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:29.797129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:29.797129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:29.797129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:29.798197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:29.798197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:29.798197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:29.798197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:29.798197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:29.798197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:29.798197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 19:49:29.798197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 19:49:29.798197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 19:49:29.798197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 19:49:29.798197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:29.798197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:29.798197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:29.798197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:29.798197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:29.798197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:29.799106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:29.799106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:29.799106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:29.799106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:29.799106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:29.799106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:29.799106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:29.799106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:29.799106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:29.799106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:29.799106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 19:49:29.799106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:29.799106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 19:49:29.799106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:29.799106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 502/1009] 10.222.0.44 () {56 vars in 1126 bytes} [Thu Jun 23 19:49:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:49:29.805293 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e304835a-ff63-499b-9fc7-0bfe51502220 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:29.835991 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf94ced-ba56-4dc8-bb71-d6202c911261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:29.836806 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf94ced-ba56-4dc8-bb71-d6202c911261 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:29.837083 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf94ced-ba56-4dc8-bb71-d6202c911261 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:29.837358 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf94ced-ba56-4dc8-bb71-d6202c911261 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:29.837997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf94ced-ba56-4dc8-bb71-d6202c911261 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:29.839369 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf94ced-ba56-4dc8-bb71-d6202c911261 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:29.852059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-2bf94ced-ba56-4dc8-bb71-d6202c911261 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 19:49:29.852059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:29.852059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 19:49:29.852059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 19:49:29.852059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 19:49:29.852059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 19:49:29.852059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 19:49:29.852059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:29.852059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 19:49:29.852059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:29.852059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:49:29.852059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:29.852059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:29.852059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:29.852059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:29.852059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:29.852059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:29.853274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:29.853274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:29.853274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:29.853274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:29.853274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:29.853274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:29.853274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 19:49:29.853274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 19:49:29.853274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 19:49:29.853274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 19:49:29.853274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:29.853274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:29.853274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:29.853274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:29.853274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:29.853274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:29.854160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:29.854160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:29.854160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:29.854160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:29.854160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:29.854160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:29.854160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:29.854160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:29.854160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:29.854160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:29.854160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 19:49:29.854160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:29.854160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 19:49:29.854160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:29.854160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 508/1010] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:49:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:49:29.859097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf94ced-ba56-4dc8-bb71-d6202c911261 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:29.880191 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:29.918305 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e304835a-ff63-499b-9fc7-0bfe51502220 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:29.959795 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb03d0-b374-4ef4-9499-c1e58f918704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:29.960485 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb03d0-b374-4ef4-9499-c1e58f918704 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:29.960775 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb03d0-b374-4ef4-9499-c1e58f918704 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:29.961053 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb03d0-b374-4ef4-9499-c1e58f918704 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:29.961756 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb03d0-b374-4ef4-9499-c1e58f918704 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:29.962067 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb03d0-b374-4ef4-9499-c1e58f918704 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:29.963120 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf94ced-ba56-4dc8-bb71-d6202c911261 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:29.963433 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb03d0-b374-4ef4-9499-c1e58f918704 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:29.972696 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 503/1011] 10.222.0.44 () {56 vars in 1122 bytes} [Thu Jun 23 19:49:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:29.976903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb03d0-b374-4ef4-9499-c1e58f918704 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:30.006020 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9736227b-7fea-413e-acb8-f48423ce1b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:30.006704 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736227b-7fea-413e-acb8-f48423ce1b32 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:30.006981 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736227b-7fea-413e-acb8-f48423ce1b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:30.007251 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736227b-7fea-413e-acb8-f48423ce1b32 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:30.007946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9736227b-7fea-413e-acb8-f48423ce1b32 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:30.008253 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9736227b-7fea-413e-acb8-f48423ce1b32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:30.009650 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9736227b-7fea-413e-acb8-f48423ce1b32 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:30.019132 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 509/1012] 10.222.0.44 () {56 vars in 1112 bytes} [Thu Jun 23 19:49:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:30.020093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9736227b-7fea-413e-acb8-f48423ce1b32 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:30.023924 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:30.092428 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb03d0-b374-4ef4-9499-c1e58f918704 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:30.124909 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:30.129785 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9736227b-7fea-413e-acb8-f48423ce1b32 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:30.130651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1510966-bb8a-4303-8dd1-b374ec1a7d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:30.131343 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1510966-bb8a-4303-8dd1-b374ec1a7d22 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:30.131620 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1510966-bb8a-4303-8dd1-b374ec1a7d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:30.131893 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1510966-bb8a-4303-8dd1-b374ec1a7d22 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:30.132634 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1510966-bb8a-4303-8dd1-b374ec1a7d22 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:30.132945 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1510966-bb8a-4303-8dd1-b374ec1a7d22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:30.134313 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1510966-bb8a-4303-8dd1-b374ec1a7d22 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:30.145753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 504/1013] 10.222.0.44 () {56 vars in 1122 bytes} [Thu Jun 23 19:49:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:30.147014 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1510966-bb8a-4303-8dd1-b374ec1a7d22 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:30.176600 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d924bf7-c702-4840-859b-5ea948a4fc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:30.177501 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d924bf7-c702-4840-859b-5ea948a4fc75 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:30.177947 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d924bf7-c702-4840-859b-5ea948a4fc75 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:30.178225 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d924bf7-c702-4840-859b-5ea948a4fc75 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:30.178931 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d924bf7-c702-4840-859b-5ea948a4fc75 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:30.179234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d924bf7-c702-4840-859b-5ea948a4fc75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:30.180646 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d924bf7-c702-4840-859b-5ea948a4fc75 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:30.189993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 510/1014] 10.222.0.44 () {56 vars in 1112 bytes} [Thu Jun 23 19:49:30 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:30.195752 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d924bf7-c702-4840-859b-5ea948a4fc75 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:30.258371 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1510966-bb8a-4303-8dd1-b374ec1a7d22 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:30.270574 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:30.293796 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b264474-3df5-4a79-a897-18eff190136f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:30.293796 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b264474-3df5-4a79-a897-18eff190136f None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:30.294165 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b264474-3df5-4a79-a897-18eff190136f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:30.294295 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b264474-3df5-4a79-a897-18eff190136f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:30.295031 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b264474-3df5-4a79-a897-18eff190136f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:30.296542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b264474-3df5-4a79-a897-18eff190136f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:30.306786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d924bf7-c702-4840-859b-5ea948a4fc75 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:30.340263 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef0882-ba1a-4144-a494-aee636701848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:30.340941 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef0882-ba1a-4144-a494-aee636701848 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:30.341225 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef0882-ba1a-4144-a494-aee636701848 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:30.341502 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef0882-ba1a-4144-a494-aee636701848 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:30.342230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef0882-ba1a-4144-a494-aee636701848 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:30.343607 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef0882-ba1a-4144-a494-aee636701848 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:30.386746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 505/1015] 10.222.0.44 () {60 vars in 1144 bytes} [Thu Jun 23 19:49:30 2022] POST /identity/v3/endpoints => generated 325 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 19:49:30.425807 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 511/1016] 10.222.0.44 () {60 vars in 1144 bytes} [Thu Jun 23 19:49:30 2022] POST /identity/v3/endpoints => generated 334 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 19:49:31.072380 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde628f1-abb9-487c-837b-0f86a727e22e None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:31.072895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde628f1-abb9-487c-837b-0f86a727e22e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:31.072972 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde628f1-abb9-487c-837b-0f86a727e22e None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:31.074438 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 506/1017] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:31.079347 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c285a-8d1a-4dd8-a1d1-691c5df0bb83 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:31.079716 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c285a-8d1a-4dd8-a1d1-691c5df0bb83 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:31.079898 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c285a-8d1a-4dd8-a1d1-691c5df0bb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:31.096974 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-4b2c285a-8d1a-4dd8-a1d1-691c5df0bb83 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:31.098200 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4b2c285a-8d1a-4dd8-a1d1-691c5df0bb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:31.184999 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 512/1018] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:31 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:31.208109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde628f1-abb9-487c-837b-0f86a727e22e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:31.271735 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:31.316968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde628f1-abb9-487c-837b-0f86a727e22e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:31.349747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad20dbe-cb2b-49dc-8a57-5b8c2373c952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:31.350465 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad20dbe-cb2b-49dc-8a57-5b8c2373c952 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:31.350744 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad20dbe-cb2b-49dc-8a57-5b8c2373c952 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:31.351014 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad20dbe-cb2b-49dc-8a57-5b8c2373c952 None admin] PATH_INFO: `/v3/services/image` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:31.351658 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad20dbe-cb2b-49dc-8a57-5b8c2373c952 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:31.353046 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad20dbe-cb2b-49dc-8a57-5b8c2373c952 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:31.364919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-6ad20dbe-cb2b-49dc-8a57-5b8c2373c952 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 19:49:31.364919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:31.364919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 19:49:31.364919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 19:49:31.364919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 19:49:31.364919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 19:49:31.364919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 19:49:31.364919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:31.364919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 19:49:31.364919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:31.364919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:49:31.364919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:31.364919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:31.364919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:31.364919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:31.364919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:31.364919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:31.365884 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:31.365884 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:31.365884 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:31.365884 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:31.365884 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:31.365884 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:31.365884 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 19:49:31.365884 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 19:49:31.365884 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 19:49:31.365884 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 19:49:31.365884 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:31.365884 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:31.365884 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:31.365884 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:31.365884 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:31.365884 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:31.366761 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:31.366761 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:31.366761 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:31.366761 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:31.366761 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:31.366761 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:31.366761 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:31.366761 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:31.366761 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:31.366761 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:31.366761 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 19:49:31.366761 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:31.366761 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 19:49:31.366761 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:31.368273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 507/1019] 10.222.0.44 () {56 vars in 1105 bytes} [Thu Jun 23 19:49:31 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:31.371961 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c285a-8d1a-4dd8-a1d1-691c5df0bb83 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:31.423224 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:31.475195 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c285a-8d1a-4dd8-a1d1-691c5df0bb83 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:31.509585 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfebf84-0b49-4dd6-ab7b-4d9da248cfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:31.510344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfebf84-0b49-4dd6-ab7b-4d9da248cfe3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:31.510632 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfebf84-0b49-4dd6-ab7b-4d9da248cfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:31.510902 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfebf84-0b49-4dd6-ab7b-4d9da248cfe3 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:31.511613 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfebf84-0b49-4dd6-ab7b-4d9da248cfe3 None admin] RBAC: Adding query filter params (name=image) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:31.511921 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfebf84-0b49-4dd6-ab7b-4d9da248cfe3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:31.513340 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfebf84-0b49-4dd6-ab7b-4d9da248cfe3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:31.523541 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 513/1020] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:31.524586 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad20dbe-cb2b-49dc-8a57-5b8c2373c952 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:31.630048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad20dbe-cb2b-49dc-8a57-5b8c2373c952 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:31.652010 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:31.666763 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bff3e2-f314-482c-a7d2-79e30f8bd062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:31.667475 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bff3e2-f314-482c-a7d2-79e30f8bd062 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:31.667755 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bff3e2-f314-482c-a7d2-79e30f8bd062 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:31.668030 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bff3e2-f314-482c-a7d2-79e30f8bd062 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:31.668758 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bff3e2-f314-482c-a7d2-79e30f8bd062 None admin] RBAC: Adding query filter params (type=image) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:31.669063 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bff3e2-f314-482c-a7d2-79e30f8bd062 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:31.670452 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bff3e2-f314-482c-a7d2-79e30f8bd062 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:31.675955 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 508/1021] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:31 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:31.681521 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfebf84-0b49-4dd6-ab7b-4d9da248cfe3 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:31.784223 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfebf84-0b49-4dd6-ab7b-4d9da248cfe3 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:31.798493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:31.818865 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242a123-dffa-44d3-8e02-426d6d6e222b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:31.819512 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242a123-dffa-44d3-8e02-426d6d6e222b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:31.819790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242a123-dffa-44d3-8e02-426d6d6e222b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:31.820060 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242a123-dffa-44d3-8e02-426d6d6e222b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:31.820868 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242a123-dffa-44d3-8e02-426d6d6e222b None admin] RBAC: Adding query filter params (interface=public, service_id=99cfd9c0d9b341daae18dbbdcce90e3d, region_id=RegionOne) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:31.821177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242a123-dffa-44d3-8e02-426d6d6e222b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:31.822546 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242a123-dffa-44d3-8e02-426d6d6e222b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:31.832586 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 514/1022] 10.222.0.44 () {56 vars in 1251 bytes} [Thu Jun 23 19:49:31 2022] GET /identity/v3/endpoints?service_id=99cfd9c0d9b341daae18dbbdcce90e3d&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:31.833468 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bff3e2-f314-482c-a7d2-79e30f8bd062 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:31.932438 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bff3e2-f314-482c-a7d2-79e30f8bd062 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:31.965658 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1752d3e-fb84-413e-96c4-96999fd42804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:31.966321 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1752d3e-fb84-413e-96c4-96999fd42804 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:31.966614 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1752d3e-fb84-413e-96c4-96999fd42804 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:31.966870 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1752d3e-fb84-413e-96c4-96999fd42804 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:31.967544 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1752d3e-fb84-413e-96c4-96999fd42804 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:31.969003 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1752d3e-fb84-413e-96c4-96999fd42804 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:31.984573 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 509/1023] 10.222.0.44 () {56 vars in 1087 bytes} [Thu Jun 23 19:49:31 2022] GET /identity/v3/services => generated 1957 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 19:49:32.458653 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b783443-f121-47bd-a191-a340e9d36ce5 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:32.459264 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b783443-f121-47bd-a191-a340e9d36ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:32.459264 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b783443-f121-47bd-a191-a340e9d36ce5 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:32.460805 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 515/1024] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:32.465737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e95d3-c5e2-4f55-af31-f93b065f7a26 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:32.466125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e95d3-c5e2-4f55-af31-f93b065f7a26 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:32.466309 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e95d3-c5e2-4f55-af31-f93b065f7a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:32.484544 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a48e95d3-c5e2-4f55-af31-f93b065f7a26 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:32.486598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a48e95d3-c5e2-4f55-af31-f93b065f7a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:32.533508 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:32.556372 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 510/1025] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:32.579219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e95d3-c5e2-4f55-af31-f93b065f7a26 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:32.579380 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07a3ef-ddda-40ef-9074-f7c1618484f4 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:32.579607 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07a3ef-ddda-40ef-9074-f7c1618484f4 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:32.579790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07a3ef-ddda-40ef-9074-f7c1618484f4 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:32.581183 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 516/1026] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:32.586622 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e5454-c76e-417f-b9a2-4fa99de66d02 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:32.586913 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e5454-c76e-417f-b9a2-4fa99de66d02 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:32.587195 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e5454-c76e-417f-b9a2-4fa99de66d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:32.603492 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-814e5454-c76e-417f-b9a2-4fa99de66d02 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:32.604702 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-814e5454-c76e-417f-b9a2-4fa99de66d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:32.673725 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 517/1027] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:32 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 19:49:32.689039 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e95d3-c5e2-4f55-af31-f93b065f7a26 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:32.698100 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e5454-c76e-417f-b9a2-4fa99de66d02 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:32.726175 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5474213d-bb46-4016-bdb1-2759478b6e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:32.726971 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5474213d-bb46-4016-bdb1-2759478b6e88 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:32.727248 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5474213d-bb46-4016-bdb1-2759478b6e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:32.727516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5474213d-bb46-4016-bdb1-2759478b6e88 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:32.728716 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5474213d-bb46-4016-bdb1-2759478b6e88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:32.730144 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5474213d-bb46-4016-bdb1-2759478b6e88 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:32.732057 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 511/1028] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:32.737411 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5474213d-bb46-4016-bdb1-2759478b6e88 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:32.788899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:32.806782 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e5454-c76e-417f-b9a2-4fa99de66d02 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:32.816463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:32.841595 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd28171-6076-4a48-84cd-4d988d2ddef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:32.843761 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd28171-6076-4a48-84cd-4d988d2ddef7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:32.843761 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd28171-6076-4a48-84cd-4d988d2ddef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:32.843761 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd28171-6076-4a48-84cd-4d988d2ddef7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:32.846184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5474213d-bb46-4016-bdb1-2759478b6e88 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:32.854400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd28171-6076-4a48-84cd-4d988d2ddef7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:49:32.863234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-8dd28171-6076-4a48-84cd-4d988d2ddef7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 19:49:32.863234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:32.863234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:32.863234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:32.863234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:32.863234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:32.863234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:32.863234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:32.863234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:32.863234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:32.863234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:32.863234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:32.863234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:49:32.863234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:49:32.863234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:49:32.863234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:49:32.864262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:49:32.864262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:49:32.864262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:49:32.864262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:49:32.864262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:32.864262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:32.864262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:32.864262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:32.864262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:32.864262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:32.864262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:32.864262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:32.864262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:32.864262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:32.864262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:32.864262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:32.865843 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:32.865843 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:32.865843 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:32.865843 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:32.865843 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:49:32.865843 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:49:32.865843 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:49:32.865843 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:49:32.865843 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:49:32.865843 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:49:32.865843 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 19:49:32.865843 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:32.865843 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 518/1029] 10.222.0.44 () {56 vars in 1120 bytes} [Thu Jun 23 19:49:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:49:32.872023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd28171-6076-4a48-84cd-4d988d2ddef7 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:32.886180 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476437b6-17b9-4c62-b22b-a3810125e3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:32.886928 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476437b6-17b9-4c62-b22b-a3810125e3d1 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:32.887272 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476437b6-17b9-4c62-b22b-a3810125e3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:32.887611 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476437b6-17b9-4c62-b22b-a3810125e3d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:32.888406 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476437b6-17b9-4c62-b22b-a3810125e3d1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:32.889899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476437b6-17b9-4c62-b22b-a3810125e3d1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:32.971004 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd28171-6076-4a48-84cd-4d988d2ddef7 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:32.988335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 512/1030] 10.222.0.44 () {60 vars in 1141 bytes} [Thu Jun 23 19:49:32 2022] POST /identity/v3/projects => generated 311 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 19:49:33.018532 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf49f5a-6a7d-4594-8087-5507a7edcbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:33.019804 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf49f5a-6a7d-4594-8087-5507a7edcbf9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:33.020083 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf49f5a-6a7d-4594-8087-5507a7edcbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:33.020403 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf49f5a-6a7d-4594-8087-5507a7edcbf9 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:33.021221 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf49f5a-6a7d-4594-8087-5507a7edcbf9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:33.021542 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf49f5a-6a7d-4594-8087-5507a7edcbf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:33.022972 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf49f5a-6a7d-4594-8087-5507a7edcbf9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:33.037326 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 519/1031] 10.222.0.44 () {56 vars in 1115 bytes} [Thu Jun 23 19:49:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:33.038736 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476437b6-17b9-4c62-b22b-a3810125e3d1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:33.145804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:33.150917 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476437b6-17b9-4c62-b22b-a3810125e3d1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:33.187225 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d7af1-4db1-46b1-84ac-13d1fc427007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:33.188337 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d7af1-4db1-46b1-84ac-13d1fc427007 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:33.188618 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d7af1-4db1-46b1-84ac-13d1fc427007 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:33.188892 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d7af1-4db1-46b1-84ac-13d1fc427007 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:33.199646 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d7af1-4db1-46b1-84ac-13d1fc427007 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:33.201137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d7af1-4db1-46b1-84ac-13d1fc427007 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:33.219641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-fc1d7af1-4db1-46b1-84ac-13d1fc427007 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:33.219641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:33.219641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:33.219641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:33.219641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:33.219641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:33.219641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:33.219641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:33.219641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:33.219641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:33.219641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:33.219641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:33.219641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:49:33.219641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:49:33.219641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:49:33.220663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:49:33.220663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:33.220663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:33.220663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:33.220663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:33.220663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:33.220663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:33.220663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:33.220663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:33.220663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:33.220663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:33.220663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:33.220663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:33.220663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:33.220663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:33.220663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:33.220663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:33.221550 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:49:33.221550 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:49:33.221550 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:33.221550 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:33.221550 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 513/1032] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:33.229277 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf49f5a-6a7d-4594-8087-5507a7edcbf9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:33.261034 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:33.330560 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf49f5a-6a7d-4594-8087-5507a7edcbf9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:33.364126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04dac6d-f373-480d-966d-c2613ad75ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:33.364838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04dac6d-f373-480d-966d-c2613ad75ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:33.365124 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04dac6d-f373-480d-966d-c2613ad75ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:33.365400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04dac6d-f373-480d-966d-c2613ad75ad6 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:33.366119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04dac6d-f373-480d-966d-c2613ad75ad6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:33.366427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04dac6d-f373-480d-966d-c2613ad75ad6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:33.367805 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04dac6d-f373-480d-966d-c2613ad75ad6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:33.380355 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 520/1033] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:33.385677 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d7af1-4db1-46b1-84ac-13d1fc427007 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:33.490283 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d7af1-4db1-46b1-84ac-13d1fc427007 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:33.524580 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7050d4-aec0-475b-adc7-35c7f4b5f09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:33.525371 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7050d4-aec0-475b-adc7-35c7f4b5f09a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:33.525653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7050d4-aec0-475b-adc7-35c7f4b5f09a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:33.525924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7050d4-aec0-475b-adc7-35c7f4b5f09a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:33.533141 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7050d4-aec0-475b-adc7-35c7f4b5f09a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:33.534626 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7050d4-aec0-475b-adc7-35c7f4b5f09a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:33.548907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-df7050d4-aec0-475b-adc7-35c7f4b5f09a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 19:49:33.548907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:33.548907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:33.548907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:33.548907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:33.548907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:33.548907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:33.548907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:33.548907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:33.548907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:33.548907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:33.548907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:33.548907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:49:33.548907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:49:33.548907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:49:33.548907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:49:33.549862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:33.549862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:33.549862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:49:33.549862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:49:33.549862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:49:33.549862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:49:33.549862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:33.549862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:33.549862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:33.549862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:33.549862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:33.549862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:33.549862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:33.549862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:33.549862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:33.549862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:33.549862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:33.549862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:33.551323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:33.551323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:33.551323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:49:33.551323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:49:33.551323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:49:33.551323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:49:33.551323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:49:33.551323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:49:33.551323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 19:49:33.551323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:33.555389 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 514/1034] 10.222.0.44 () {56 vars in 1093 bytes} [Thu Jun 23 19:49:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:33.556231 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04dac6d-f373-480d-966d-c2613ad75ad6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:33.646496 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:33.656451 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04dac6d-f373-480d-966d-c2613ad75ad6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:33.691857 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dff232-a7a8-4c62-a173-defa4711885e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:33.692567 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dff232-a7a8-4c62-a173-defa4711885e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:33.692855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dff232-a7a8-4c62-a173-defa4711885e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:33.693131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dff232-a7a8-4c62-a173-defa4711885e None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:33.693965 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dff232-a7a8-4c62-a173-defa4711885e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:33.694276 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dff232-a7a8-4c62-a173-defa4711885e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:33.695716 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dff232-a7a8-4c62-a173-defa4711885e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:33.717997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 521/1035] 10.222.0.44 () {56 vars in 1133 bytes} [Thu Jun 23 19:49:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:33.719438 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7050d4-aec0-475b-adc7-35c7f4b5f09a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:33.732173 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:33.820147 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7050d4-aec0-475b-adc7-35c7f4b5f09a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:33.853520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114fd5e2-39e3-4d38-a664-10301e995eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:33.854160 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114fd5e2-39e3-4d38-a664-10301e995eec None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:33.854445 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114fd5e2-39e3-4d38-a664-10301e995eec None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:33.854723 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114fd5e2-39e3-4d38-a664-10301e995eec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:33.921024 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114fd5e2-39e3-4d38-a664-10301e995eec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:33.922490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114fd5e2-39e3-4d38-a664-10301e995eec None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:33.933366 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:34.017545 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6d985-395a-4592-91ef-30887752426f None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:34.017833 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6d985-395a-4592-91ef-30887752426f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:34.018108 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6d985-395a-4592-91ef-30887752426f None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:34.020030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 522/1036] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:34.026427 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 515/1037] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:49:33 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 19:49:34.026594 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6135b5f-5fb0-43be-bbde-aa69519b65c5 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:34.026594 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6135b5f-5fb0-43be-bbde-aa69519b65c5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:34.026594 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6135b5f-5fb0-43be-bbde-aa69519b65c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:34.027453 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114fd5e2-39e3-4d38-a664-10301e995eec None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:34.041772 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f6135b5f-5fb0-43be-bbde-aa69519b65c5 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:34.042952 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f6135b5f-5fb0-43be-bbde-aa69519b65c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:34.110044 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 523/1038] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:34 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 19:49:34.134463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6135b5f-5fb0-43be-bbde-aa69519b65c5 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:34.143717 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114fd5e2-39e3-4d38-a664-10301e995eec None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:34.155207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:34.180693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31692d2e-0e00-4031-9d76-f91e1f820b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:34.181494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31692d2e-0e00-4031-9d76-f91e1f820b49 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:34.181776 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31692d2e-0e00-4031-9d76-f91e1f820b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:34.182048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31692d2e-0e00-4031-9d76-f91e1f820b49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:34.192954 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31692d2e-0e00-4031-9d76-f91e1f820b49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:34.194412 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31692d2e-0e00-4031-9d76-f91e1f820b49 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:34.212712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-31692d2e-0e00-4031-9d76-f91e1f820b49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:34.212712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:34.212712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:34.212712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:34.212712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:34.212712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:34.212712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:34.212712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:34.212712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:34.212712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:34.212712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:34.212712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:34.212712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:49:34.212712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:49:34.212712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:49:34.213609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:49:34.213609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:34.213609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:34.213609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:34.213609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:34.213609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:34.213609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:34.213609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:34.213609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:34.213609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:34.213609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:34.213609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:34.213609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:34.213609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:34.213609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:34.213609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:34.213609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:34.214659 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:49:34.214659 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:49:34.214659 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:34.214659 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:34.218397 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 516/1039] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:34.219340 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31692d2e-0e00-4031-9d76-f91e1f820b49 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:34.220621 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:34.249176 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6135b5f-5fb0-43be-bbde-aa69519b65c5 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:34.284250 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b329a1-1cf8-4079-b392-89d141e903d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:34.285103 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b329a1-1cf8-4079-b392-89d141e903d9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:34.285384 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b329a1-1cf8-4079-b392-89d141e903d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:34.285682 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b329a1-1cf8-4079-b392-89d141e903d9 None admin] PATH_INFO: `/v3/services/image` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:34.286304 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b329a1-1cf8-4079-b392-89d141e903d9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:34.287675 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b329a1-1cf8-4079-b392-89d141e903d9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:34.299864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-21b329a1-1cf8-4079-b392-89d141e903d9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 19:49:34.299864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:34.299864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 19:49:34.299864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 19:49:34.299864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 19:49:34.299864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 19:49:34.299864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 19:49:34.299864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:34.299864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 19:49:34.299864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:34.299864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:49:34.299864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:34.299864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:34.299864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:34.299864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:34.299864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:34.299864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:34.301429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:34.301429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:34.301429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:34.301429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:34.301429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:34.301429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:34.301429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 19:49:34.301429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 19:49:34.301429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 19:49:34.301429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 19:49:34.301429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:34.301429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:34.301429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:34.301429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:34.301429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:34.301429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:34.302288 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:34.302288 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:34.302288 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:34.302288 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:34.302288 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:34.302288 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:34.302288 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:34.302288 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:34.302288 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:34.302288 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:34.302288 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 19:49:34.302288 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:49:34.302288 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 19:49:34.302288 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:34.306584 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 524/1040] 10.222.0.44 () {56 vars in 1105 bytes} [Thu Jun 23 19:49:34 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:34.307318 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b329a1-1cf8-4079-b392-89d141e903d9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:34.329636 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31692d2e-0e00-4031-9d76-f91e1f820b49 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:34.366042 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f8016-8001-450f-931c-bcb224911268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:34.366620 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f8016-8001-450f-931c-bcb224911268 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:34.366895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f8016-8001-450f-931c-bcb224911268 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:34.367168 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f8016-8001-450f-931c-bcb224911268 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:34.367876 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f8016-8001-450f-931c-bcb224911268 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:34.368178 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f8016-8001-450f-931c-bcb224911268 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:34.369564 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f8016-8001-450f-931c-bcb224911268 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:34.380809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 517/1041] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:49:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:34.386013 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f8016-8001-450f-931c-bcb224911268 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:34.410225 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:34.412015 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b329a1-1cf8-4079-b392-89d141e903d9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:34.450427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d54ee-036a-4974-b233-b046b5548604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:34.451117 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d54ee-036a-4974-b233-b046b5548604 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:34.451401 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d54ee-036a-4974-b233-b046b5548604 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:34.451675 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746d54ee-036a-4974-b233-b046b5548604 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:34.452402 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d54ee-036a-4974-b233-b046b5548604 None admin] RBAC: Adding query filter params (name=image) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:34.452711 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d54ee-036a-4974-b233-b046b5548604 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:34.454083 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746d54ee-036a-4974-b233-b046b5548604 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:34.463452 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 525/1042] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:34.464505 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d54ee-036a-4974-b233-b046b5548604 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:34.475267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:34.494448 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f8016-8001-450f-931c-bcb224911268 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:34.534315 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a670a-0d4d-48f8-a8b2-4f4bebed7f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:34.535627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a670a-0d4d-48f8-a8b2-4f4bebed7f45 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:34.535902 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a670a-0d4d-48f8-a8b2-4f4bebed7f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:34.536170 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a670a-0d4d-48f8-a8b2-4f4bebed7f45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:34.543000 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a670a-0d4d-48f8-a8b2-4f4bebed7f45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:34.544406 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a670a-0d4d-48f8-a8b2-4f4bebed7f45 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:34.558471 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-8e2a670a-0d4d-48f8-a8b2-4f4bebed7f45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:49:34.558471 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:34.558471 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:34.558471 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:34.558471 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:34.558471 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:34.558471 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:34.558471 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:34.558471 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:34.558471 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:34.558471 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:34.558471 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:34.558471 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:49:34.558471 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:49:34.558471 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:49:34.559489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:49:34.559489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:34.559489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:34.559489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:34.559489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:34.559489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:34.559489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:34.559489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:34.559489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:34.559489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:34.559489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:34.559489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:34.559489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:34.559489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:34.559489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:34.559489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:34.559489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:34.560893 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:49:34.560893 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:49:34.560893 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:49:34.560893 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:49:34.560893 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:49:34.560893 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:49:34.560893 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:49:34.560893 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:34.564784 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 518/1043] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:49:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:49:34.565916 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a670a-0d4d-48f8-a8b2-4f4bebed7f45 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:34.574635 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746d54ee-036a-4974-b233-b046b5548604 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:34.609841 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98233caa-f265-4e21-bc2c-3a5b3df4f9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:34.610508 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98233caa-f265-4e21-bc2c-3a5b3df4f9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:34.610788 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98233caa-f265-4e21-bc2c-3a5b3df4f9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:34.611064 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98233caa-f265-4e21-bc2c-3a5b3df4f9c6 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:34.611770 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98233caa-f265-4e21-bc2c-3a5b3df4f9c6 None admin] RBAC: Adding query filter params (type=image) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:34.612075 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98233caa-f265-4e21-bc2c-3a5b3df4f9c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:34.614070 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98233caa-f265-4e21-bc2c-3a5b3df4f9c6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:34.619309 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 526/1044] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:34 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:34.624538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98233caa-f265-4e21-bc2c-3a5b3df4f9c6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:34.656594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:34.675439 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a670a-0d4d-48f8-a8b2-4f4bebed7f45 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:34.693013 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:34.714832 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d2ee90-6706-411e-8b81-b9d0e04254d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:34.715804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d2ee90-6706-411e-8b81-b9d0e04254d7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:34.716266 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d2ee90-6706-411e-8b81-b9d0e04254d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:34.716692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d2ee90-6706-411e-8b81-b9d0e04254d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:34.717682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d2ee90-6706-411e-8b81-b9d0e04254d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:34.717995 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d2ee90-6706-411e-8b81-b9d0e04254d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:34.719973 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d2ee90-6706-411e-8b81-b9d0e04254d7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:34.724355 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98233caa-f265-4e21-bc2c-3a5b3df4f9c6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:34.734228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 519/1045] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:49:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:34.740184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d2ee90-6706-411e-8b81-b9d0e04254d7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:34.762960 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ff459-edcd-4fde-bd34-22c735bff633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:34.763921 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ff459-edcd-4fde-bd34-22c735bff633 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:34.764216 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ff459-edcd-4fde-bd34-22c735bff633 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:34.764508 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ff459-edcd-4fde-bd34-22c735bff633 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:34.765248 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ff459-edcd-4fde-bd34-22c735bff633 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:34.766629 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ff459-edcd-4fde-bd34-22c735bff633 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:34.838859 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 527/1046] 10.222.0.44 () {60 vars in 1144 bytes} [Thu Jun 23 19:49:34 2022] POST /identity/v3/endpoints => generated 327 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 19:49:34.844238 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d2ee90-6706-411e-8b81-b9d0e04254d7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:34.891119 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3eb40-0c3d-44cf-928c-99a82d07605d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:34.891753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3eb40-0c3d-44cf-928c-99a82d07605d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:34.892037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3eb40-0c3d-44cf-928c-99a82d07605d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:34.892330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3eb40-0c3d-44cf-928c-99a82d07605d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:34.893194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd3eb40-0c3d-44cf-928c-99a82d07605d None admin] RBAC: Adding query filter params (role.id=ec091d6061664ed697565c76e216b5b4, scope.project.id=155180ba63c446ca889a6eb3f94cc31a, user.id=a63d1d902c024c97a38f7d92f73814d3) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:34.893556 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd3eb40-0c3d-44cf-928c-99a82d07605d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:34.895076 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd3eb40-0c3d-44cf-928c-99a82d07605d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:34.908656 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 520/1047] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:49:34 2022] GET /identity/v3/role_assignments?user.id=a63d1d902c024c97a38f7d92f73814d3&scope.project.id=155180ba63c446ca889a6eb3f94cc31a&role.id=ec091d6061664ed697565c76e216b5b4 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:35.066973 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9face-9774-44ce-b6e2-0b148fdded46 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:35.067262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9face-9774-44ce-b6e2-0b148fdded46 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:35.067552 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9face-9774-44ce-b6e2-0b148fdded46 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:35.068950 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 528/1048] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:35.074173 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c5025-1d6d-4b5b-90b7-d386c96663ed None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:35.074465 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c5025-1d6d-4b5b-90b7-d386c96663ed None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:35.074749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c5025-1d6d-4b5b-90b7-d386c96663ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:35.080089 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:35.092064 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e49c5025-1d6d-4b5b-90b7-d386c96663ed None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:35.093286 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e49c5025-1d6d-4b5b-90b7-d386c96663ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:35.164261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 521/1049] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:35.189362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9face-9774-44ce-b6e2-0b148fdded46 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:35.299992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9face-9774-44ce-b6e2-0b148fdded46 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:35.326374 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:35.334906 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929886d-3ed8-4ba9-99f0-f068aed9bcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:35.335701 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929886d-3ed8-4ba9-99f0-f068aed9bcdd None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:35.335985 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929886d-3ed8-4ba9-99f0-f068aed9bcdd None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:35.336260 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929886d-3ed8-4ba9-99f0-f068aed9bcdd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:35.337473 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2929886d-3ed8-4ba9-99f0-f068aed9bcdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:35.338929 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2929886d-3ed8-4ba9-99f0-f068aed9bcdd None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:35.344429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 529/1050] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:35.346734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c5025-1d6d-4b5b-90b7-d386c96663ed None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:35.452017 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c5025-1d6d-4b5b-90b7-d386c96663ed None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:35.483148 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:35.489135 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843bf0ea-e0ea-41cc-beaf-9138e2842e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:35.489805 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bf0ea-e0ea-41cc-beaf-9138e2842e97 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:35.490150 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bf0ea-e0ea-41cc-beaf-9138e2842e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:35.490422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bf0ea-e0ea-41cc-beaf-9138e2842e97 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:35.491131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843bf0ea-e0ea-41cc-beaf-9138e2842e97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:35.492616 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843bf0ea-e0ea-41cc-beaf-9138e2842e97 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:35.606326 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 522/1051] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jun 23 19:49:35 2022] POST /identity/v3/users => generated 285 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 19:49:36.907097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6e703-80ad-49c4-ad89-5ec6772b6e5a None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:36.907650 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6e703-80ad-49c4-ad89-5ec6772b6e5a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:36.907650 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6e703-80ad-49c4-ad89-5ec6772b6e5a None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:36.909190 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 530/1052] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:36.914679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d544197-5e87-4bef-b8f3-238a4af4d6c6 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:36.915052 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d544197-5e87-4bef-b8f3-238a4af4d6c6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:36.915228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d544197-5e87-4bef-b8f3-238a4af4d6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:36.932127 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9d544197-5e87-4bef-b8f3-238a4af4d6c6 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:36.933341 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9d544197-5e87-4bef-b8f3-238a4af4d6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:36.936832 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47127aa-8cd6-4c15-9771-d1f4e39be847 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:36.937119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47127aa-8cd6-4c15-9771-d1f4e39be847 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:36.937659 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47127aa-8cd6-4c15-9771-d1f4e39be847 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:36.939154 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 531/1053] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:49:36.943839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5536e0a-b59c-44d6-a0d7-d32009d6b551 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:36.944126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5536e0a-b59c-44d6-a0d7-d32009d6b551 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:36.944430 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5536e0a-b59c-44d6-a0d7-d32009d6b551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:36.960822 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c5536e0a-b59c-44d6-a0d7-d32009d6b551 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:36.961997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c5536e0a-b59c-44d6-a0d7-d32009d6b551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:37.005474 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 523/1054] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:37.027769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d544197-5e87-4bef-b8f3-238a4af4d6c6 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:37.033360 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 532/1055] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 19:49:37.056867 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5536e0a-b59c-44d6-a0d7-d32009d6b551 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:37.138211 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d544197-5e87-4bef-b8f3-238a4af4d6c6 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:37.160651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:37.162083 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5536e0a-b59c-44d6-a0d7-d32009d6b551 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:37.177587 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec5877-5884-4945-b33f-378199c1a66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:37.178238 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec5877-5884-4945-b33f-378199c1a66c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:37.178522 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec5877-5884-4945-b33f-378199c1a66c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:37.178797 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec5877-5884-4945-b33f-378199c1a66c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:37.186219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:37.197936 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a9f72-0bc8-43fe-aff9-e14779eb3af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:37.198744 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a9f72-0bc8-43fe-aff9-e14779eb3af7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:37.199110 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a9f72-0bc8-43fe-aff9-e14779eb3af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:37.199392 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a9f72-0bc8-43fe-aff9-e14779eb3af7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:37.211361 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a9f72-0bc8-43fe-aff9-e14779eb3af7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:49:37.219615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-1c0a9f72-0bc8-43fe-aff9-e14779eb3af7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 19:49:37.219615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:37.219615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:37.219615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:37.219615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:37.219615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:37.219615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:37.219615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:37.219615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:37.219615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:37.219615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:37.219615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:37.219615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:49:37.219615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:49:37.219615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:49:37.219615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:49:37.220721 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:49:37.220721 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:49:37.220721 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:49:37.220721 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:49:37.220721 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:37.220721 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:37.220721 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:37.220721 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:37.220721 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:37.220721 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:37.220721 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:37.220721 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:37.220721 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:37.220721 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:37.220721 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:37.220721 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:37.221563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:37.221563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:37.221563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:37.221563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:37.221563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:49:37.221563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:49:37.221563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:49:37.221563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:49:37.221563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:49:37.221563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:49:37.221563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 19:49:37.221563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:37.228575 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 533/1056] 10.222.0.44 () {56 vars in 1120 bytes} [Thu Jun 23 19:49:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:49:37.228575 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a9f72-0bc8-43fe-aff9-e14779eb3af7 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:37.258203 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ec5877-5884-4945-b33f-378199c1a66c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:37.259766 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ec5877-5884-4945-b33f-378199c1a66c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:37.334968 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a9f72-0bc8-43fe-aff9-e14779eb3af7 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:37.367307 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 524/1057] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:49:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 19:49:37.369771 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed91320-f49a-40b9-8e0c-4567e66b9184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:37.370434 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed91320-f49a-40b9-8e0c-4567e66b9184 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:37.370712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed91320-f49a-40b9-8e0c-4567e66b9184 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:37.370980 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed91320-f49a-40b9-8e0c-4567e66b9184 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:37.371812 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed91320-f49a-40b9-8e0c-4567e66b9184 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:37.372123 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed91320-f49a-40b9-8e0c-4567e66b9184 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:37.372894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec5877-5884-4945-b33f-378199c1a66c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:37.373529 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed91320-f49a-40b9-8e0c-4567e66b9184 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:37.382859 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 534/1058] 10.222.0.44 () {56 vars in 1115 bytes} [Thu Jun 23 19:49:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:37.388141 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed91320-f49a-40b9-8e0c-4567e66b9184 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:37.394356 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:37.437895 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:37.481663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec5877-5884-4945-b33f-378199c1a66c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:37.491895 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed91320-f49a-40b9-8e0c-4567e66b9184 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:37.516111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e3977-a1b1-49cd-a9af-c8e287f0cd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:37.516945 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e3977-a1b1-49cd-a9af-c8e287f0cd75 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:37.517228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e3977-a1b1-49cd-a9af-c8e287f0cd75 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:37.517553 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e3977-a1b1-49cd-a9af-c8e287f0cd75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:37.527263 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f120f54-34b8-47b9-a75c-6c6da4dc12a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:37.528564 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e3977-a1b1-49cd-a9af-c8e287f0cd75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:37.528681 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f120f54-34b8-47b9-a75c-6c6da4dc12a7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:37.529110 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f120f54-34b8-47b9-a75c-6c6da4dc12a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:37.529585 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f120f54-34b8-47b9-a75c-6c6da4dc12a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:37.530354 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e3977-a1b1-49cd-a9af-c8e287f0cd75 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:37.544516 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f120f54-34b8-47b9-a75c-6c6da4dc12a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:37.547228 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f120f54-34b8-47b9-a75c-6c6da4dc12a7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:37.552021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-f05e3977-a1b1-49cd-a9af-c8e287f0cd75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:37.552021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:37.552021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:37.552021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:37.552021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:37.552021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:37.552021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:37.552021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:37.552021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:37.552021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:37.552021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:37.552021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:37.552021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:49:37.552021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:49:37.552021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:49:37.553074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:49:37.553074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:37.553074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:37.553074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:37.553074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:37.553074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:37.553074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:37.553074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:37.553074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:37.553074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:37.553074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:37.553074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:37.553074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:37.553074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:37.553074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:37.553074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:37.553074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:37.554463 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:49:37.554463 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:49:37.554463 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:37.554463 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:37.556659 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 525/1059] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:37.558936 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e3977-a1b1-49cd-a9af-c8e287f0cd75 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:37.567422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-8f120f54-34b8-47b9-a75c-6c6da4dc12a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:37.567422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:37.567422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:37.567422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:37.567422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:37.567422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:37.567422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:37.567422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:37.567422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:37.567422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:37.567422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:37.567422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:37.567422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:49:37.567422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:49:37.567422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:49:37.568365 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:49:37.568365 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:37.568365 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:37.568365 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:37.568365 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:37.568365 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:37.568365 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:37.568365 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:37.568365 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:37.568365 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:37.568365 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:37.568365 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:37.568365 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:37.568365 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:37.568365 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:37.568365 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:37.568365 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:37.569447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:49:37.569447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:49:37.569447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:37.569447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:37.569447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 535/1060] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:49:37.574143 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f120f54-34b8-47b9-a75c-6c6da4dc12a7 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:37.633663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:37.667280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e3977-a1b1-49cd-a9af-c8e287f0cd75 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:37.671386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:37.675270 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f120f54-34b8-47b9-a75c-6c6da4dc12a7 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:37.705896 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc99025-e80f-4735-bc9a-20b9d4071511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:37.706535 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc99025-e80f-4735-bc9a-20b9d4071511 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:37.706819 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc99025-e80f-4735-bc9a-20b9d4071511 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:37.707099 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc99025-e80f-4735-bc9a-20b9d4071511 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:37.707813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc99025-e80f-4735-bc9a-20b9d4071511 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:37.708119 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc99025-e80f-4735-bc9a-20b9d4071511 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:37.709550 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc99025-e80f-4735-bc9a-20b9d4071511 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:37.711362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f18fe-b292-4195-984b-82a58fbeba6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:37.712031 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f18fe-b292-4195-984b-82a58fbeba6e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:37.712330 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f18fe-b292-4195-984b-82a58fbeba6e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:37.712616 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f18fe-b292-4195-984b-82a58fbeba6e None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:37.713341 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f18fe-b292-4195-984b-82a58fbeba6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:37.713650 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f18fe-b292-4195-984b-82a58fbeba6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:37.715021 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f18fe-b292-4195-984b-82a58fbeba6e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:37.728358 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 536/1061] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:37.732591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 526/1062] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:37.733981 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f18fe-b292-4195-984b-82a58fbeba6e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:37.751839 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2ff7c-d980-404f-9a74-a29cd38a0fd3 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:37.752135 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2ff7c-d980-404f-9a74-a29cd38a0fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:37.752452 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2ff7c-d980-404f-9a74-a29cd38a0fd3 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:37.754958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 527/1063] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:49:37.760534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c433cc-0dbb-4551-af4f-516bb6486fc0 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:37.760841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c433cc-0dbb-4551-af4f-516bb6486fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:37.761107 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c433cc-0dbb-4551-af4f-516bb6486fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:37.782600 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-22c433cc-0dbb-4551-af4f-516bb6486fc0 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:37.783811 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-22c433cc-0dbb-4551-af4f-516bb6486fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:37.856792 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f18fe-b292-4195-984b-82a58fbeba6e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:37.869661 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 528/1064] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 19:49:37.893030 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c433cc-0dbb-4551-af4f-516bb6486fc0 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:37.898468 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0ca1f-159a-449f-b655-47d60a3c4cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:37.899598 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0ca1f-159a-449f-b655-47d60a3c4cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:37.901163 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0ca1f-159a-449f-b655-47d60a3c4cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:37.901163 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0ca1f-159a-449f-b655-47d60a3c4cf6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:37.909365 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f0ca1f-159a-449f-b655-47d60a3c4cf6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:37.911940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f0ca1f-159a-449f-b655-47d60a3c4cf6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:37.927926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-f9f0ca1f-159a-449f-b655-47d60a3c4cf6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 19:49:37.927926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:37.927926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:37.927926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:37.927926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:37.927926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:37.927926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:37.927926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:37.927926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:37.927926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:37.927926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:37.927926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:37.927926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:49:37.927926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:49:37.927926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:49:37.927926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:49:37.929226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:37.929226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:37.929226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:49:37.929226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:49:37.929226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:49:37.929226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:49:37.929226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:37.929226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:37.929226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:37.929226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:37.929226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:37.929226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:37.929226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:37.929226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:37.929226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:37.929226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:37.929226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:37.929226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:37.930480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:37.930480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:37.930480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:49:37.930480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:49:37.930480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:49:37.930480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:49:37.930480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:49:37.930480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:49:37.930480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 19:49:37.930480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:37.930480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 537/1065] 10.222.0.44 () {56 vars in 1093 bytes} [Thu Jun 23 19:49:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:37.935020 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0ca1f-159a-449f-b655-47d60a3c4cf6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:37.953827 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:37.960784 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:38.007544 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c433cc-0dbb-4551-af4f-516bb6486fc0 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:38.043546 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4136e5-4df5-4833-bc1a-534ed16bb04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:38.044372 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4136e5-4df5-4833-bc1a-534ed16bb04a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:38.044497 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0ca1f-159a-449f-b655-47d60a3c4cf6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:38.044654 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4136e5-4df5-4833-bc1a-534ed16bb04a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:38.044930 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4136e5-4df5-4833-bc1a-534ed16bb04a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:38.055020 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4136e5-4df5-4833-bc1a-534ed16bb04a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:49:38.062746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-db4136e5-4df5-4833-bc1a-534ed16bb04a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:49:38.062746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:38.062746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:38.062746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:38.062746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:38.062746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:38.062746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:38.062746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:38.062746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:38.062746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:38.062746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:38.062746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:38.062746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:49:38.062746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:49:38.062746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:49:38.062746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:49:38.063905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:49:38.063905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:49:38.063905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:49:38.063905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:49:38.063905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:38.063905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:38.063905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:38.063905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:38.063905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:38.063905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:38.063905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:38.063905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:38.063905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:38.063905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:38.063905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:38.063905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:38.064873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:38.064873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:38.064873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:38.064873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:38.064873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:49:38.064873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:49:38.064873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:49:38.064873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:49:38.064873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:49:38.064873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:49:38.064873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:49:38.064873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:38.064873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 529/1066] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:49:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:49:38.070042 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4136e5-4df5-4833-bc1a-534ed16bb04a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:38.079434 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348f611-444c-4e72-8bd0-bf003be83492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:38.080104 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348f611-444c-4e72-8bd0-bf003be83492 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:38.080421 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348f611-444c-4e72-8bd0-bf003be83492 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:38.080701 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348f611-444c-4e72-8bd0-bf003be83492 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:38.082077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9348f611-444c-4e72-8bd0-bf003be83492 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:38.082388 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9348f611-444c-4e72-8bd0-bf003be83492 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:38.083838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9348f611-444c-4e72-8bd0-bf003be83492 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:38.101377 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 538/1067] 10.222.0.44 () {56 vars in 1133 bytes} [Thu Jun 23 19:49:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:38.106702 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348f611-444c-4e72-8bd0-bf003be83492 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:38.167660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4136e5-4df5-4833-bc1a-534ed16bb04a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:38.179359 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:38.186534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:38.201326 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458960c-4799-4b17-9a39-c053926588a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:38.201933 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348f611-444c-4e72-8bd0-bf003be83492 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:38.202051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458960c-4799-4b17-9a39-c053926588a0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:38.202256 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458960c-4799-4b17-9a39-c053926588a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:38.202532 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458960c-4799-4b17-9a39-c053926588a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:38.203280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e458960c-4799-4b17-9a39-c053926588a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:38.203582 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e458960c-4799-4b17-9a39-c053926588a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:38.204989 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e458960c-4799-4b17-9a39-c053926588a0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:38.213654 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 530/1068] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:49:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:38.219204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458960c-4799-4b17-9a39-c053926588a0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:38.238923 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532262f8-2379-434c-ad0b-a1f758bdd148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:38.239716 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532262f8-2379-434c-ad0b-a1f758bdd148 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:38.239995 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532262f8-2379-434c-ad0b-a1f758bdd148 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:38.240261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532262f8-2379-434c-ad0b-a1f758bdd148 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:38.250915 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532262f8-2379-434c-ad0b-a1f758bdd148 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:38.252408 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532262f8-2379-434c-ad0b-a1f758bdd148 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:38.269271 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-532262f8-2379-434c-ad0b-a1f758bdd148 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:38.269271 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:38.269271 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:38.269271 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:38.269271 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:38.269271 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:38.269271 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:38.269271 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:38.269271 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:38.269271 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:38.269271 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:38.269271 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:38.269271 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:49:38.269271 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:49:38.269271 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:49:38.270362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:49:38.270362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:38.270362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:38.270362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:38.270362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:38.270362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:38.270362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:38.270362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:38.270362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:38.270362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:38.270362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:38.270362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:38.270362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:38.270362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:38.270362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:38.270362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:38.270362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:38.271449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:49:38.271449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:49:38.271449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:38.271449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:38.271449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 539/1069] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:49:38.276007 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532262f8-2379-434c-ad0b-a1f758bdd148 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:38.321463 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458960c-4799-4b17-9a39-c053926588a0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:38.355649 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bba424-7c9c-42d4-9577-f8c9b2177d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:38.356537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bba424-7c9c-42d4-9577-f8c9b2177d69 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:38.356823 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bba424-7c9c-42d4-9577-f8c9b2177d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:38.357099 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bba424-7c9c-42d4-9577-f8c9b2177d69 None admin] PATH_INFO: `/v3/users/45b0f6f000994bc49eca5536f253c4b4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:38.371178 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bba424-7c9c-42d4-9577-f8c9b2177d69 None admin] RBAC: Authorizing `identity:get_user(user_id=45b0f6f000994bc49eca5536f253c4b4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:38.372451 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532262f8-2379-434c-ad0b-a1f758bdd148 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:38.372642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bba424-7c9c-42d4-9577-f8c9b2177d69 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:38.374323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 531/1070] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:49:38 2022] GET /identity/v3/users/45b0f6f000994bc49eca5536f253c4b4 => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:38.380368 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bba424-7c9c-42d4-9577-f8c9b2177d69 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:38.398233 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:38.405867 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d1325-df8b-4992-b2af-a6a70470b465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:38.406519 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d1325-df8b-4992-b2af-a6a70470b465 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:38.406801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d1325-df8b-4992-b2af-a6a70470b465 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:38.407074 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d1325-df8b-4992-b2af-a6a70470b465 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:38.407809 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d1325-df8b-4992-b2af-a6a70470b465 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:38.408096 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d1325-df8b-4992-b2af-a6a70470b465 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:38.410011 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d1325-df8b-4992-b2af-a6a70470b465 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:38.421033 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 540/1071] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:49:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:38.423134 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:38.426914 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d1325-df8b-4992-b2af-a6a70470b465 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:38.482666 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bba424-7c9c-42d4-9577-f8c9b2177d69 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:38.515749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a04d28-e469-4029-85bb-16047c3d9239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:38.516417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a04d28-e469-4029-85bb-16047c3d9239 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:38.516702 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a04d28-e469-4029-85bb-16047c3d9239 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:38.516973 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a04d28-e469-4029-85bb-16047c3d9239 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:38.522731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d1325-df8b-4992-b2af-a6a70470b465 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:38.556510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef5195d-e6c5-40c3-aa56-4c88bf6e732e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:38.557286 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5195d-e6c5-40c3-aa56-4c88bf6e732e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:38.557564 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5195d-e6c5-40c3-aa56-4c88bf6e732e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:38.557835 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5195d-e6c5-40c3-aa56-4c88bf6e732e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:38.564840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef5195d-e6c5-40c3-aa56-4c88bf6e732e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:38.566347 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef5195d-e6c5-40c3-aa56-4c88bf6e732e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:38.580006 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-fef5195d-e6c5-40c3-aa56-4c88bf6e732e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:49:38.580006 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:38.580006 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:38.580006 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:38.580006 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:38.580006 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:38.580006 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:38.580006 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:38.580006 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:38.580006 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:38.580006 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:38.580006 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:38.580006 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:49:38.580006 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:49:38.580006 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:49:38.581046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:49:38.581046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:38.581046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:38.581046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:38.581046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:38.581046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:38.581046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:38.581046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:38.581046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:38.581046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:38.581046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:38.581046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:38.581046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:38.581046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:38.581046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:38.581046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:38.581046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:38.582486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:49:38.582486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:49:38.582486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:49:38.582486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:49:38.582486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:49:38.582486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:49:38.582486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:49:38.582486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:38.582486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 541/1072] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:49:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:49:38.587359 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef5195d-e6c5-40c3-aa56-4c88bf6e732e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:38.593509 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a04d28-e469-4029-85bb-16047c3d9239 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:38.594950 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a04d28-e469-4029-85bb-16047c3d9239 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:38.633548 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:38.635969 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:38.683893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef5195d-e6c5-40c3-aa56-4c88bf6e732e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:38.690873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 532/1073] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:49:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 19:49:38.696400 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a04d28-e469-4029-85bb-16047c3d9239 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:38.718040 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb8d3a-a4b6-43c3-8b44-676ea7a5a3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:38.718691 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb8d3a-a4b6-43c3-8b44-676ea7a5a3c2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:38.718970 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb8d3a-a4b6-43c3-8b44-676ea7a5a3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:38.719246 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb8d3a-a4b6-43c3-8b44-676ea7a5a3c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:38.720018 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb8d3a-a4b6-43c3-8b44-676ea7a5a3c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:38.720341 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb8d3a-a4b6-43c3-8b44-676ea7a5a3c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:38.721903 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb8d3a-a4b6-43c3-8b44-676ea7a5a3c2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:38.733487 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 542/1074] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:49:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:38.739091 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb8d3a-a4b6-43c3-8b44-676ea7a5a3c2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:38.795410 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a04d28-e469-4029-85bb-16047c3d9239 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:38.829571 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1db6c-ec97-4917-a248-f0e9aab77eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:38.830412 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1db6c-ec97-4917-a248-f0e9aab77eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:38.830692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1db6c-ec97-4917-a248-f0e9aab77eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:38.830965 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1db6c-ec97-4917-a248-f0e9aab77eb0 None admin] PATH_INFO: `/v3/projects/ce5366140a8d43b3bce1dc77ef8e6859` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:38.832132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1db6c-ec97-4917-a248-f0e9aab77eb0 None admin] RBAC: Authorizing `identity:get_project(project_id=ce5366140a8d43b3bce1dc77ef8e6859)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:38.833588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1db6c-ec97-4917-a248-f0e9aab77eb0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:38.835281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 533/1075] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:49:38 2022] GET /identity/v3/projects/ce5366140a8d43b3bce1dc77ef8e6859 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:38.835413 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb8d3a-a4b6-43c3-8b44-676ea7a5a3c2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:38.841129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1db6c-ec97-4917-a248-f0e9aab77eb0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:38.844276 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:38.851358 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:38.868381 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7c2f4-0471-4bf7-90a8-73fb85500f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:38.869268 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7c2f4-0471-4bf7-90a8-73fb85500f9d None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:38.869554 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7c2f4-0471-4bf7-90a8-73fb85500f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:38.869875 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7c2f4-0471-4bf7-90a8-73fb85500f9d None admin] PATH_INFO: `/v3/projects/155180ba63c446ca889a6eb3f94cc31a/users/a63d1d902c024c97a38f7d92f73814d3/roles/ec091d6061664ed697565c76e216b5b4` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:38.872167 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7c2f4-0471-4bf7-90a8-73fb85500f9d None admin] RBAC: Authorizing `identity:create_grant(project_id=155180ba63c446ca889a6eb3f94cc31a, user_id=a63d1d902c024c97a38f7d92f73814d3, role_id=ec091d6061664ed697565c76e216b5b4)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:38.873641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7c2f4-0471-4bf7-90a8-73fb85500f9d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:38.935119 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1db6c-ec97-4917-a248-f0e9aab77eb0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:38.936490 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 543/1076] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jun 23 19:49:38 2022] PUT /identity/v3/projects/155180ba63c446ca889a6eb3f94cc31a/users/a63d1d902c024c97a38f7d92f73814d3/roles/ec091d6061664ed697565c76e216b5b4 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 19:49:38.974628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16cce1b-0315-4e68-99e8-7c6185cacc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:38.975292 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cce1b-0315-4e68-99e8-7c6185cacc3e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:38.975579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cce1b-0315-4e68-99e8-7c6185cacc3e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:38.975855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cce1b-0315-4e68-99e8-7c6185cacc3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:38.976758 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16cce1b-0315-4e68-99e8-7c6185cacc3e None admin] RBAC: Adding query filter params (role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb, scope.project.id=ce5366140a8d43b3bce1dc77ef8e6859, user.id=45b0f6f000994bc49eca5536f253c4b4) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:38.977078 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16cce1b-0315-4e68-99e8-7c6185cacc3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:38.978530 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16cce1b-0315-4e68-99e8-7c6185cacc3e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:38.985098 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 534/1077] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:49:38 2022] GET /identity/v3/role_assignments?user.id=45b0f6f000994bc49eca5536f253c4b4&scope.project.id=ce5366140a8d43b3bce1dc77ef8e6859&role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:40.980410 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd89fa-16aa-4481-adba-9b27988fa6a7 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:40.980978 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd89fa-16aa-4481-adba-9b27988fa6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:40.980978 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd89fa-16aa-4481-adba-9b27988fa6a7 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:40.982319 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 544/1078] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:40.987668 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543498e-28c8-4871-ba25-e8497e09a6c8 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:40.988091 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543498e-28c8-4871-ba25-e8497e09a6c8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:40.988221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543498e-28c8-4871-ba25-e8497e09a6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:41.003939 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4efcb2-4de7-439a-95b0-c23e7f7e3e49 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:41.004221 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4efcb2-4de7-439a-95b0-c23e7f7e3e49 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:41.004523 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4efcb2-4de7-439a-95b0-c23e7f7e3e49 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:41.005683 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a543498e-28c8-4871-ba25-e8497e09a6c8 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:41.006001 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 545/1079] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:49:41.006879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a543498e-28c8-4871-ba25-e8497e09a6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:41.010700 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be419c-dd18-41a7-bfd9-3909f4185c39 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:41.010980 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be419c-dd18-41a7-bfd9-3909f4185c39 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:41.011251 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be419c-dd18-41a7-bfd9-3909f4185c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:41.027573 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-38be419c-dd18-41a7-bfd9-3909f4185c39 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:41.028781 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-38be419c-dd18-41a7-bfd9-3909f4185c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:41.073605 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 535/1080] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:41.098830 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543498e-28c8-4871-ba25-e8497e09a6c8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:41.100472 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 546/1081] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 19:49:41.101127 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:41.126004 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be419c-dd18-41a7-bfd9-3909f4185c39 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:41.215059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543498e-28c8-4871-ba25-e8497e09a6c8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:41.236926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:41.241194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be419c-dd18-41a7-bfd9-3909f4185c39 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:41.254908 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c24d8e-9e87-4580-80ba-c4e567c75422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:41.255855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c24d8e-9e87-4580-80ba-c4e567c75422 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:41.256134 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c24d8e-9e87-4580-80ba-c4e567c75422 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:41.256435 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c24d8e-9e87-4580-80ba-c4e567c75422 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:41.267735 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c24d8e-9e87-4580-80ba-c4e567c75422 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:49:41.274614 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4965c72e-45c1-4e01-ba00-4a66c14d87e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:41.275402 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965c72e-45c1-4e01-ba00-4a66c14d87e1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:41.275684 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965c72e-45c1-4e01-ba00-4a66c14d87e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:41.275862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-f2c24d8e-9e87-4580-80ba-c4e567c75422 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 19:49:41.275862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:41.275862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:41.275862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:41.275862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:41.275862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:41.275862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:41.275862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:41.275862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:41.275862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:41.275862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:41.275862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:41.275862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:49:41.275862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:49:41.275862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:49:41.275862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:49:41.276782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:49:41.276782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:49:41.276782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:49:41.276782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:49:41.276782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:41.276782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:41.276782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:41.276782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:41.276782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:41.276782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:41.276782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:41.276782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:41.276782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:41.276782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:41.276782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:41.276782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:41.277845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:41.277845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:41.277845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:41.277845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:41.277845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:49:41.277845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:49:41.277845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:49:41.277845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:49:41.277845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:49:41.277845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:49:41.277845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 19:49:41.277845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:41.278859 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4965c72e-45c1-4e01-ba00-4a66c14d87e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:41.278934 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 536/1082] 10.222.0.44 () {56 vars in 1120 bytes} [Thu Jun 23 19:49:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:49:41.283118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c24d8e-9e87-4580-80ba-c4e567c75422 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:41.289540 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4965c72e-45c1-4e01-ba00-4a66c14d87e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:49:41.297240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-4965c72e-45c1-4e01-ba00-4a66c14d87e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:49:41.297240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:41.297240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:41.297240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:41.297240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:41.297240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:41.297240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:41.297240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:41.297240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:41.297240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:41.297240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:41.297240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:41.297240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:49:41.297240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:49:41.297240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:49:41.297240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:49:41.298244 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:49:41.298244 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:49:41.298244 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:49:41.298244 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:49:41.298244 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:41.298244 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:41.298244 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:41.298244 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:41.298244 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:41.298244 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:41.298244 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:41.298244 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:41.298244 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:41.298244 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:41.298244 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:41.298244 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:41.299150 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:41.299150 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:41.299150 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:41.299150 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:41.299150 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:49:41.299150 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:49:41.299150 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:49:41.299150 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:49:41.299150 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:49:41.299150 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:49:41.299150 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:49:41.299150 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:41.300246 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 547/1083] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:49:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:49:41.304641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4965c72e-45c1-4e01-ba00-4a66c14d87e1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:41.350177 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:41.392021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c24d8e-9e87-4580-80ba-c4e567c75422 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:41.410758 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4965c72e-45c1-4e01-ba00-4a66c14d87e1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:41.427532 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d3e790-ef20-4046-ad1f-afa1e046dbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:41.430584 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3e790-ef20-4046-ad1f-afa1e046dbe2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:41.430584 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3e790-ef20-4046-ad1f-afa1e046dbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:41.430584 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3e790-ef20-4046-ad1f-afa1e046dbe2 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:41.430584 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d3e790-ef20-4046-ad1f-afa1e046dbe2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:41.430584 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d3e790-ef20-4046-ad1f-afa1e046dbe2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:41.431125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d3e790-ef20-4046-ad1f-afa1e046dbe2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:41.441723 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 537/1084] 10.222.0.44 () {56 vars in 1115 bytes} [Thu Jun 23 19:49:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:41.447303 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d3e790-ef20-4046-ad1f-afa1e046dbe2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:41.447493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f4381-2a69-4e06-b13f-f9d6c8f95796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:41.448161 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f4381-2a69-4e06-b13f-f9d6c8f95796 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:41.448469 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f4381-2a69-4e06-b13f-f9d6c8f95796 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:41.448742 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f4381-2a69-4e06-b13f-f9d6c8f95796 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:41.449650 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f4381-2a69-4e06-b13f-f9d6c8f95796 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:41.449846 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f4381-2a69-4e06-b13f-f9d6c8f95796 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:41.451220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8f4381-2a69-4e06-b13f-f9d6c8f95796 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:41.460574 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 548/1085] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:49:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:41.466892 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f4381-2a69-4e06-b13f-f9d6c8f95796 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:41.493165 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:41.567407 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d3e790-ef20-4046-ad1f-afa1e046dbe2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:41.583598 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f4381-2a69-4e06-b13f-f9d6c8f95796 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:41.590603 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:41.602812 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0f6e6-5ee4-49be-a540-6a64d8962a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:41.603695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0f6e6-5ee4-49be-a540-6a64d8962a01 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:41.603898 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0f6e6-5ee4-49be-a540-6a64d8962a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:41.604164 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0f6e6-5ee4-49be-a540-6a64d8962a01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:41.616054 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0f6e6-5ee4-49be-a540-6a64d8962a01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:41.617527 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e0f6e6-5ee4-49be-a540-6a64d8962a01 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:41.619913 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9eedb-e7db-45ba-9d15-ad9ba9152467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:41.620809 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9eedb-e7db-45ba-9d15-ad9ba9152467 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:41.621093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9eedb-e7db-45ba-9d15-ad9ba9152467 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:41.621564 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e9eedb-e7db-45ba-9d15-ad9ba9152467 None admin] PATH_INFO: `/v3/users/45b0f6f000994bc49eca5536f253c4b4` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:41.622771 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9eedb-e7db-45ba-9d15-ad9ba9152467 None admin] RBAC: Authorizing `identity:get_user(user_id=45b0f6f000994bc49eca5536f253c4b4)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:41.624210 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e9eedb-e7db-45ba-9d15-ad9ba9152467 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:41.626160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 549/1086] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:49:41 2022] GET /identity/v3/users/45b0f6f000994bc49eca5536f253c4b4 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:41.632845 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9eedb-e7db-45ba-9d15-ad9ba9152467 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:41.639174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-e2e0f6e6-5ee4-49be-a540-6a64d8962a01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:41.639174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:41.639174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:41.639174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:41.639174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:41.639174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:41.639174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:41.639174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:41.639174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:41.639174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:41.639174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:41.639174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:41.639174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:49:41.639174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:49:41.639174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:49:41.640267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:49:41.640267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:41.640267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:41.640267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:41.640267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:41.640267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:41.640267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:41.640267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:41.640267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:41.640267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:41.640267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:41.640267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:41.640267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:41.640267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:41.640267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:41.640267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:41.640267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:41.641555 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:49:41.641555 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:49:41.641555 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:41.641555 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:41.641555 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 538/1087] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:49:41.646079 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0f6e6-5ee4-49be-a540-6a64d8962a01 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:41.733775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:41.746456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e9eedb-e7db-45ba-9d15-ad9ba9152467 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:41.753712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0f6e6-5ee4-49be-a540-6a64d8962a01 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:41.781489 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004666f2-d7fb-4aab-87ce-101bbb3c2338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:41.782570 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004666f2-d7fb-4aab-87ce-101bbb3c2338 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:41.782914 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004666f2-d7fb-4aab-87ce-101bbb3c2338 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:41.783250 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004666f2-d7fb-4aab-87ce-101bbb3c2338 None admin] PATH_INFO: `/v3/projects/ce5366140a8d43b3bce1dc77ef8e6859` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:41.785618 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004666f2-d7fb-4aab-87ce-101bbb3c2338 None admin] RBAC: Authorizing `identity:get_project(project_id=ce5366140a8d43b3bce1dc77ef8e6859)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:41.787079 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004666f2-d7fb-4aab-87ce-101bbb3c2338 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:41.787828 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537cab6a-683a-4782-b82b-8762f58864cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:41.788540 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537cab6a-683a-4782-b82b-8762f58864cc None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:41.789042 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537cab6a-683a-4782-b82b-8762f58864cc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:41.789152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 550/1088] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:49:41 2022] GET /identity/v3/projects/ce5366140a8d43b3bce1dc77ef8e6859 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:41.789249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537cab6a-683a-4782-b82b-8762f58864cc None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:41.789892 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537cab6a-683a-4782-b82b-8762f58864cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:41.790279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537cab6a-683a-4782-b82b-8762f58864cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:41.791705 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537cab6a-683a-4782-b82b-8762f58864cc None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:41.794886 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004666f2-d7fb-4aab-87ce-101bbb3c2338 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:41.805163 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 539/1089] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:41.813034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537cab6a-683a-4782-b82b-8762f58864cc None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:41.851022 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:41.903680 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004666f2-d7fb-4aab-87ce-101bbb3c2338 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:41.923389 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537cab6a-683a-4782-b82b-8762f58864cc None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:41.941416 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e9e18-6e08-4132-9ff2-aacc08d14d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:41.942818 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e9e18-6e08-4132-9ff2-aacc08d14d48 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:41.943327 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e9e18-6e08-4132-9ff2-aacc08d14d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:41.943828 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e9e18-6e08-4132-9ff2-aacc08d14d48 None admin] PATH_INFO: `/v3/projects/ce5366140a8d43b3bce1dc77ef8e6859/users/45b0f6f000994bc49eca5536f253c4b4/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:41.947070 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e9e18-6e08-4132-9ff2-aacc08d14d48 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce5366140a8d43b3bce1dc77ef8e6859, user_id=45b0f6f000994bc49eca5536f253c4b4, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:41.948562 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e9e18-6e08-4132-9ff2-aacc08d14d48 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:41.962060 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e6f19-1ae4-4173-983b-94cf88e774af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:41.962896 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e6f19-1ae4-4173-983b-94cf88e774af None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:41.963192 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e6f19-1ae4-4173-983b-94cf88e774af None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:41.963460 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e6f19-1ae4-4173-983b-94cf88e774af None admin] PATH_INFO: `/v3/users/nova` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:41.971191 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e6f19-1ae4-4173-983b-94cf88e774af None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:41.973186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e6f19-1ae4-4173-983b-94cf88e774af None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:41.988044 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-b43e6f19-1ae4-4173-983b-94cf88e774af None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 19:49:41.988044 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:41.988044 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:41.988044 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:41.988044 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:41.988044 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:41.988044 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:41.988044 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:41.988044 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:41.988044 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:41.988044 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:41.988044 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:41.988044 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:49:41.988044 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:49:41.988044 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:49:41.988044 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:49:41.989593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:41.989593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:41.989593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:49:41.989593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:49:41.989593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:49:41.989593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:49:41.989593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:41.989593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:41.989593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:41.989593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:41.989593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:41.989593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:41.989593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:41.989593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:41.989593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:41.989593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:41.989593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:41.989593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:41.990625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:41.990625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:41.990625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:49:41.990625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:49:41.990625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:49:41.990625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:49:41.990625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:49:41.990625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:49:41.990625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 19:49:41.990625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:41.990625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 540/1090] 10.222.0.44 () {56 vars in 1093 bytes} [Thu Jun 23 19:49:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:41.995099 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e6f19-1ae4-4173-983b-94cf88e774af None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:42.009220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 551/1091] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jun 23 19:49:41 2022] PUT /identity/v3/projects/ce5366140a8d43b3bce1dc77ef8e6859/users/45b0f6f000994bc49eca5536f253c4b4/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 19:49:42.092848 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:42.099119 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e6f19-1ae4-4173-983b-94cf88e774af None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:42.134417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041b263-6be3-484c-9e69-622aa410865a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:42.135089 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041b263-6be3-484c-9e69-622aa410865a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:42.135373 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041b263-6be3-484c-9e69-622aa410865a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:42.135641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041b263-6be3-484c-9e69-622aa410865a None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:42.136531 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c041b263-6be3-484c-9e69-622aa410865a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:42.136836 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c041b263-6be3-484c-9e69-622aa410865a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:42.138270 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c041b263-6be3-484c-9e69-622aa410865a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:42.161751 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 541/1092] 10.222.0.44 () {56 vars in 1133 bytes} [Thu Jun 23 19:49:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:49:42.167491 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e9e18-6e08-4132-9ff2-aacc08d14d48 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:42.186628 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:42.267864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e9e18-6e08-4132-9ff2-aacc08d14d48 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:42.301233 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba83e40-2bd5-4db1-b71d-fc4fa311f0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:42.302185 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba83e40-2bd5-4db1-b71d-fc4fa311f0f4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:42.302621 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba83e40-2bd5-4db1-b71d-fc4fa311f0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:42.302925 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba83e40-2bd5-4db1-b71d-fc4fa311f0f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:42.369007 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba83e40-2bd5-4db1-b71d-fc4fa311f0f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:42.370475 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba83e40-2bd5-4db1-b71d-fc4fa311f0f4 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:42.386709 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:42.467730 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 552/1093] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:49:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 19:49:42.476014 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041b263-6be3-484c-9e69-622aa410865a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:42.582910 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041b263-6be3-484c-9e69-622aa410865a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:42.615409 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999285f3-a491-4441-ab43-9a118a342659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:42.616188 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999285f3-a491-4441-ab43-9a118a342659 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:42.616491 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999285f3-a491-4441-ab43-9a118a342659 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:42.616761 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999285f3-a491-4441-ab43-9a118a342659 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:42.626504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999285f3-a491-4441-ab43-9a118a342659 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:42.628517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999285f3-a491-4441-ab43-9a118a342659 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:42.645349 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-999285f3-a491-4441-ab43-9a118a342659 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:42.645349 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:42.645349 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:42.645349 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:42.645349 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:42.645349 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:42.645349 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:42.645349 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:42.645349 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:42.645349 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:42.645349 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:42.645349 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:42.645349 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:49:42.645349 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:49:42.645349 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:49:42.646243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:49:42.646243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:42.646243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:42.646243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:42.646243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:42.646243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:42.646243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:42.646243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:42.646243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:42.646243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:42.646243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:42.646243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:42.646243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:42.646243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:42.646243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:42.646243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:42.646243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:42.647329 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:49:42.647329 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:49:42.647329 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:49:42.647329 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:42.647329 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 542/1094] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:42.652112 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba83e40-2bd5-4db1-b71d-fc4fa311f0f4 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:42.746883 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba83e40-2bd5-4db1-b71d-fc4fa311f0f4 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:42.763594 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:42.782118 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d8f06-8499-4d8d-974c-2f84dd8f8aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:42.792267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d8f06-8499-4d8d-974c-2f84dd8f8aef None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:42.792267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d8f06-8499-4d8d-974c-2f84dd8f8aef None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:42.792267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d8f06-8499-4d8d-974c-2f84dd8f8aef None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:42.792267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d8f06-8499-4d8d-974c-2f84dd8f8aef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:42.792267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d8f06-8499-4d8d-974c-2f84dd8f8aef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:42.792267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d8f06-8499-4d8d-974c-2f84dd8f8aef None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:42.797191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 553/1095] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:42.803612 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999285f3-a491-4441-ab43-9a118a342659 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:42.813132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:42.902019 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999285f3-a491-4441-ab43-9a118a342659 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:42.933598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b446be-2596-473e-abee-3de8c2d89e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:42.934372 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b446be-2596-473e-abee-3de8c2d89e4b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:42.934654 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b446be-2596-473e-abee-3de8c2d89e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:42.934927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b446be-2596-473e-abee-3de8c2d89e4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:42.942158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b446be-2596-473e-abee-3de8c2d89e4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:42.943611 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b446be-2596-473e-abee-3de8c2d89e4b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:42.958015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-18b446be-2596-473e-abee-3de8c2d89e4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:49:42.958015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:42.958015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:42.958015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:42.958015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:42.958015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:42.958015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:42.958015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:42.958015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:42.958015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:42.958015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:42.958015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:42.958015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:49:42.958015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:49:42.958015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:49:42.959054 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:49:42.959054 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:42.959054 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:42.959054 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:42.959054 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:42.959054 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:42.959054 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:42.959054 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:42.959054 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:42.959054 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:42.959054 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:42.959054 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:42.959054 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:42.959054 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:42.959054 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:42.959054 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:42.959054 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:42.960149 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:49:42.960149 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:49:42.960149 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:49:42.960149 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:49:42.960149 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:49:42.960149 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:49:42.960149 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:49:42.960149 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:49:42.963911 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 543/1096] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:49:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:42.965019 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d8f06-8499-4d8d-974c-2f84dd8f8aef None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:43.062077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d8f06-8499-4d8d-974c-2f84dd8f8aef None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:43.095605 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a802b-15fe-4ab9-a4e6-02e37ac87854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:43.096285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a802b-15fe-4ab9-a4e6-02e37ac87854 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:43.096628 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a802b-15fe-4ab9-a4e6-02e37ac87854 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:43.096902 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a802b-15fe-4ab9-a4e6-02e37ac87854 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:43.097690 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a802b-15fe-4ab9-a4e6-02e37ac87854 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:43.098000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a802b-15fe-4ab9-a4e6-02e37ac87854 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:43.099464 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a802b-15fe-4ab9-a4e6-02e37ac87854 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:43.113273 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 554/1097] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:49:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:43.119363 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b446be-2596-473e-abee-3de8c2d89e4b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:43.183699 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:43.217314 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b446be-2596-473e-abee-3de8c2d89e4b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:43.251918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f3e87-b3d1-4b0c-8b4b-11c19356ac8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:43.252626 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f3e87-b3d1-4b0c-8b4b-11c19356ac8b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:43.253435 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f3e87-b3d1-4b0c-8b4b-11c19356ac8b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:43.253721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f3e87-b3d1-4b0c-8b4b-11c19356ac8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:43.254543 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f3e87-b3d1-4b0c-8b4b-11c19356ac8b None admin] RBAC: Adding query filter params (role.id=ec091d6061664ed697565c76e216b5b4, scope.project.id=155180ba63c446ca889a6eb3f94cc31a, user.id=a63d1d902c024c97a38f7d92f73814d3) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:43.254856 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f3e87-b3d1-4b0c-8b4b-11c19356ac8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:43.256293 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f3e87-b3d1-4b0c-8b4b-11c19356ac8b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:43.262023 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 544/1098] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:49:43 2022] GET /identity/v3/role_assignments?user.id=a63d1d902c024c97a38f7d92f73814d3&scope.project.id=155180ba63c446ca889a6eb3f94cc31a&role.id=ec091d6061664ed697565c76e216b5b4 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:44.036126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79375a5-c521-4b1e-b348-a2f81696a90e None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:44.036449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79375a5-c521-4b1e-b348-a2f81696a90e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:44.036726 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79375a5-c521-4b1e-b348-a2f81696a90e None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:44.038085 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 555/1099] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:44.043228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddca7f-59eb-42a6-bea9-7737781a7fb3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:44.043522 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddca7f-59eb-42a6-bea9-7737781a7fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:44.043802 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddca7f-59eb-42a6-bea9-7737781a7fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:44.061389 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-33ddca7f-59eb-42a6-bea9-7737781a7fb3 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:44.062853 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-33ddca7f-59eb-42a6-bea9-7737781a7fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:44.126969 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 545/1100] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 19:49:44.150823 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79375a5-c521-4b1e-b348-a2f81696a90e None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:44.185579 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:44.248604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79375a5-c521-4b1e-b348-a2f81696a90e None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:44.281353 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476d70d-39a3-46e0-bcbf-bbc13d22e74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:44.282125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476d70d-39a3-46e0-bcbf-bbc13d22e74d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:44.282402 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476d70d-39a3-46e0-bcbf-bbc13d22e74d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:44.282669 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476d70d-39a3-46e0-bcbf-bbc13d22e74d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:44.292578 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8476d70d-39a3-46e0-bcbf-bbc13d22e74d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:49:44.300028 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-8476d70d-39a3-46e0-bcbf-bbc13d22e74d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:49:44.300028 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:49:44.300028 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:49:44.300028 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:49:44.300028 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:49:44.300028 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:49:44.300028 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:49:44.300028 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:49:44.300028 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:49:44.300028 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:49:44.300028 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:49:44.300028 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:49:44.300028 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:49:44.300028 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:49:44.300028 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:49:44.300028 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:49:44.301024 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:49:44.301024 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:49:44.301024 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:49:44.301024 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:49:44.301024 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:49:44.301024 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:49:44.301024 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:49:44.301024 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:49:44.301024 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:49:44.301024 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:49:44.301024 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:49:44.301024 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:49:44.301024 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:49:44.301024 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:49:44.301024 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:49:44.301024 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:49:44.302077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:49:44.302077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:49:44.302077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:49:44.302077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:49:44.302077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:49:44.302077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:49:44.302077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:49:44.302077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:49:44.302077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:49:44.302077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:49:44.302077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:49:44.302077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:49:44.302077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 556/1101] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:49:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:49:44.307225 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ddca7f-59eb-42a6-bea9-7737781a7fb3 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:44.364691 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:44.404546 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ddca7f-59eb-42a6-bea9-7737781a7fb3 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:44.436349 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e131a-f3d6-4d08-b185-f2441ef8b3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:44.437005 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e131a-f3d6-4d08-b185-f2441ef8b3f9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:44.437287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e131a-f3d6-4d08-b185-f2441ef8b3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:44.437816 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e131a-f3d6-4d08-b185-f2441ef8b3f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:44.438552 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e131a-f3d6-4d08-b185-f2441ef8b3f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:44.438857 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e131a-f3d6-4d08-b185-f2441ef8b3f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:44.440732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e131a-f3d6-4d08-b185-f2441ef8b3f9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:44.448950 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 546/1102] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:49:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:44.454650 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476d70d-39a3-46e0-bcbf-bbc13d22e74d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:44.547264 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:44.552776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476d70d-39a3-46e0-bcbf-bbc13d22e74d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:44.586509 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c255f37c-ff5f-4d73-a02e-5f367b0dc2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:44.587363 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c255f37c-ff5f-4d73-a02e-5f367b0dc2cd None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:44.587642 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c255f37c-ff5f-4d73-a02e-5f367b0dc2cd None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:44.587911 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c255f37c-ff5f-4d73-a02e-5f367b0dc2cd None admin] PATH_INFO: `/v3/users/45b0f6f000994bc49eca5536f253c4b4` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:44.589115 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c255f37c-ff5f-4d73-a02e-5f367b0dc2cd None admin] RBAC: Authorizing `identity:get_user(user_id=45b0f6f000994bc49eca5536f253c4b4)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:44.590556 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c255f37c-ff5f-4d73-a02e-5f367b0dc2cd None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:44.592159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 557/1103] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:49:44 2022] GET /identity/v3/users/45b0f6f000994bc49eca5536f253c4b4 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:44.598104 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e131a-f3d6-4d08-b185-f2441ef8b3f9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:44.693706 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e131a-f3d6-4d08-b185-f2441ef8b3f9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:44.715036 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:44.726020 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3b934-c5e6-4eb1-802f-20ebd93fde4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:44.726654 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3b934-c5e6-4eb1-802f-20ebd93fde4a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:44.726934 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3b934-c5e6-4eb1-802f-20ebd93fde4a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:44.731857 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3b934-c5e6-4eb1-802f-20ebd93fde4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:44.792783 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3b934-c5e6-4eb1-802f-20ebd93fde4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:44.794220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3b934-c5e6-4eb1-802f-20ebd93fde4a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:44.887538 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 547/1104] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:49:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 19:49:44.893982 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c255f37c-ff5f-4d73-a02e-5f367b0dc2cd None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:44.990681 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c255f37c-ff5f-4d73-a02e-5f367b0dc2cd None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:45.022369 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff84385-f14d-4b58-b2f6-66c2b51907a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:45.023203 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff84385-f14d-4b58-b2f6-66c2b51907a2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:45.023484 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff84385-f14d-4b58-b2f6-66c2b51907a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:45.023756 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff84385-f14d-4b58-b2f6-66c2b51907a2 None admin] PATH_INFO: `/v3/projects/ce5366140a8d43b3bce1dc77ef8e6859` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:45.024947 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff84385-f14d-4b58-b2f6-66c2b51907a2 None admin] RBAC: Authorizing `identity:get_project(project_id=ce5366140a8d43b3bce1dc77ef8e6859)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:45.026380 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff84385-f14d-4b58-b2f6-66c2b51907a2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:45.028027 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 558/1105] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:49:44 2022] GET /identity/v3/projects/ce5366140a8d43b3bce1dc77ef8e6859 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:45.034512 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3b934-c5e6-4eb1-802f-20ebd93fde4a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:45.057504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:45.129188 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3b934-c5e6-4eb1-802f-20ebd93fde4a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:45.161969 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7fc6be-d218-4d8c-ae68-d481e293c827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:45.162538 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fc6be-d218-4d8c-ae68-d481e293c827 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:45.162817 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fc6be-d218-4d8c-ae68-d481e293c827 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:45.163087 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fc6be-d218-4d8c-ae68-d481e293c827 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:45.163894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7fc6be-d218-4d8c-ae68-d481e293c827 None admin] RBAC: Adding query filter params (role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb, scope.project.id=ce5366140a8d43b3bce1dc77ef8e6859, user.id=45b0f6f000994bc49eca5536f253c4b4) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:45.164201 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7fc6be-d218-4d8c-ae68-d481e293c827 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:45.165653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7fc6be-d218-4d8c-ae68-d481e293c827 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:45.171226 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 548/1106] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:49:45 2022] GET /identity/v3/role_assignments?user.id=45b0f6f000994bc49eca5536f253c4b4&scope.project.id=ce5366140a8d43b3bce1dc77ef8e6859&role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:47.206657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ec38f-f317-4bde-a398-b77fa037efb3 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:47.207193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ec38f-f317-4bde-a398-b77fa037efb3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:47.207847 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ec38f-f317-4bde-a398-b77fa037efb3 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:47.209262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 559/1107] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:47.214351 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51218b61-14fe-4b7d-8be7-f7f401866551 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:47.214765 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51218b61-14fe-4b7d-8be7-f7f401866551 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:47.214907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51218b61-14fe-4b7d-8be7-f7f401866551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:47.231389 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-51218b61-14fe-4b7d-8be7-f7f401866551 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:47.232620 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-51218b61-14fe-4b7d-8be7-f7f401866551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:47.297159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 549/1108] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:47.321969 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ec38f-f317-4bde-a398-b77fa037efb3 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:47.352464 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:47.419565 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ec38f-f317-4bde-a398-b77fa037efb3 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:47.451359 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2d4e8-3675-4e53-8acf-16fb021cafda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:47.452130 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2d4e8-3675-4e53-8acf-16fb021cafda None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:47.452434 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2d4e8-3675-4e53-8acf-16fb021cafda None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:47.452703 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2d4e8-3675-4e53-8acf-16fb021cafda None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:47.453844 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a2d4e8-3675-4e53-8acf-16fb021cafda None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:47.455270 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a2d4e8-3675-4e53-8acf-16fb021cafda None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:47.457541 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 560/1109] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:47.463159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51218b61-14fe-4b7d-8be7-f7f401866551 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:47.479919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:47.559267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51218b61-14fe-4b7d-8be7-f7f401866551 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:47.591695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54ed7e-6cbb-4182-9f3c-c769a6801f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:47.592371 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54ed7e-6cbb-4182-9f3c-c769a6801f7f None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:47.592658 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54ed7e-6cbb-4182-9f3c-c769a6801f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:47.592928 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54ed7e-6cbb-4182-9f3c-c769a6801f7f None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:47.593622 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54ed7e-6cbb-4182-9f3c-c769a6801f7f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:47.595048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54ed7e-6cbb-4182-9f3c-c769a6801f7f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:47.701854 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 550/1110] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jun 23 19:49:47 2022] POST /identity/v3/users => generated 286 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:49:49.749372 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286873a-5da8-4454-8972-f9d73c9967ba None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:49.749894 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286873a-5da8-4454-8972-f9d73c9967ba None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:49.749987 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286873a-5da8-4454-8972-f9d73c9967ba None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:49.751296 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 561/1111] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:49.756863 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e1173-a844-4a28-a2e8-ee92bc189c6d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:49.757278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e1173-a844-4a28-a2e8-ee92bc189c6d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:49.757421 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e1173-a844-4a28-a2e8-ee92bc189c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:49.774230 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-734e1173-a844-4a28-a2e8-ee92bc189c6d None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:49.775399 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-734e1173-a844-4a28-a2e8-ee92bc189c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:49.839199 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 551/1112] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:49.862586 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286873a-5da8-4454-8972-f9d73c9967ba None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:49.955687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:49.959223 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286873a-5da8-4454-8972-f9d73c9967ba None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:49.991375 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f93a2-9127-4284-8807-52777c742428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:49.992215 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f93a2-9127-4284-8807-52777c742428 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:49.992724 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f93a2-9127-4284-8807-52777c742428 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:49.993203 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f93a2-9127-4284-8807-52777c742428 None admin] PATH_INFO: `/v3/roles/03e4cfc753754643ae08fa909647cfe3` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:49.994945 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f93a2-9127-4284-8807-52777c742428 None admin] RBAC: Authorizing `identity:get_role(role_id=03e4cfc753754643ae08fa909647cfe3)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:49.996353 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f93a2-9127-4284-8807-52777c742428 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:49.997698 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 562/1113] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:49:49 2022] GET /identity/v3/roles/03e4cfc753754643ae08fa909647cfe3 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:50.004185 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e1173-a844-4a28-a2e8-ee92bc189c6d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:50.006271 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:50.105101 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e1173-a844-4a28-a2e8-ee92bc189c6d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:50.137821 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698bccc2-5e60-46f9-b27b-ca3b06bccbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:50.138672 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bccc2-5e60-46f9-b27b-ca3b06bccbaf None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:50.138950 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bccc2-5e60-46f9-b27b-ca3b06bccbaf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:50.139221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bccc2-5e60-46f9-b27b-ca3b06bccbaf None admin] PATH_INFO: `/v3/users/1f4237db19f840f6b5228a11348be6b8` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:50.151644 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698bccc2-5e60-46f9-b27b-ca3b06bccbaf None admin] RBAC: Authorizing `identity:get_user(user_id=1f4237db19f840f6b5228a11348be6b8)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:50.153104 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698bccc2-5e60-46f9-b27b-ca3b06bccbaf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:50.154731 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 552/1114] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:49:50 2022] GET /identity/v3/users/1f4237db19f840f6b5228a11348be6b8 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:50.160334 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f93a2-9127-4284-8807-52777c742428 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:50.255926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f93a2-9127-4284-8807-52777c742428 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:50.288317 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ceea0-4912-4c78-933d-071d8c0f01e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:50.288969 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ceea0-4912-4c78-933d-071d8c0f01e5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:50.289251 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ceea0-4912-4c78-933d-071d8c0f01e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:50.289525 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ceea0-4912-4c78-933d-071d8c0f01e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:50.322152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:50.352976 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ceea0-4912-4c78-933d-071d8c0f01e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:50.354529 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ceea0-4912-4c78-933d-071d8c0f01e5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:50.448639 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 563/1115] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:49:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 19:49:50.454473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698bccc2-5e60-46f9-b27b-ca3b06bccbaf None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:50.511746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:50.557918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698bccc2-5e60-46f9-b27b-ca3b06bccbaf None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:50.594458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab84531-c089-4a3e-a34f-4e22c7f042a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:50.595550 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab84531-c089-4a3e-a34f-4e22c7f042a3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:50.595829 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab84531-c089-4a3e-a34f-4e22c7f042a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:50.596099 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab84531-c089-4a3e-a34f-4e22c7f042a3 None admin] PATH_INFO: `/v3/projects/ce5366140a8d43b3bce1dc77ef8e6859` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:50.597330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab84531-c089-4a3e-a34f-4e22c7f042a3 None admin] RBAC: Authorizing `identity:get_project(project_id=ce5366140a8d43b3bce1dc77ef8e6859)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:50.598838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab84531-c089-4a3e-a34f-4e22c7f042a3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:50.600562 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 553/1116] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:49:50 2022] GET /identity/v3/projects/ce5366140a8d43b3bce1dc77ef8e6859 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:50.606818 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ceea0-4912-4c78-933d-071d8c0f01e5 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:50.669811 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:50.702442 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ceea0-4912-4c78-933d-071d8c0f01e5 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:50.733895 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe9277-59bf-4e7f-9af6-2a020b306dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:50.734525 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe9277-59bf-4e7f-9af6-2a020b306dbc None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:50.734801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe9277-59bf-4e7f-9af6-2a020b306dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:50.735071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe9277-59bf-4e7f-9af6-2a020b306dbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:50.735880 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe9277-59bf-4e7f-9af6-2a020b306dbc None admin] RBAC: Adding query filter params (role.id=03e4cfc753754643ae08fa909647cfe3, scope.project.id=ce5366140a8d43b3bce1dc77ef8e6859, user.id=1f4237db19f840f6b5228a11348be6b8) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:50.736187 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe9277-59bf-4e7f-9af6-2a020b306dbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:50.738155 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe9277-59bf-4e7f-9af6-2a020b306dbc None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:50.743356 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 564/1117] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:49:50 2022] GET /identity/v3/role_assignments?user.id=1f4237db19f840f6b5228a11348be6b8&scope.project.id=ce5366140a8d43b3bce1dc77ef8e6859&role.id=03e4cfc753754643ae08fa909647cfe3 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:52.718557 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea6857-d58e-4a23-80a1-e2af5ad5d500 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:52.719088 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea6857-d58e-4a23-80a1-e2af5ad5d500 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:52.719088 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea6857-d58e-4a23-80a1-e2af5ad5d500 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:52.720486 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 554/1118] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:52.726086 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e0df9-e96f-43ce-a582-5b9251bee8d2 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:52.726497 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e0df9-e96f-43ce-a582-5b9251bee8d2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:52.726644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e0df9-e96f-43ce-a582-5b9251bee8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:52.743535 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-583e0df9-e96f-43ce-a582-5b9251bee8d2 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:52.744749 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-583e0df9-e96f-43ce-a582-5b9251bee8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:52.808218 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 565/1119] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:52.830495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea6857-d58e-4a23-80a1-e2af5ad5d500 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:52.933581 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea6857-d58e-4a23-80a1-e2af5ad5d500 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:52.959556 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:52.966856 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4609b-a4c9-49fa-b549-f387df03edbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:52.967740 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4609b-a4c9-49fa-b549-f387df03edbb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:52.968019 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4609b-a4c9-49fa-b549-f387df03edbb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:52.968286 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4609b-a4c9-49fa-b549-f387df03edbb None admin] PATH_INFO: `/v3/roles/03e4cfc753754643ae08fa909647cfe3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:52.969802 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4609b-a4c9-49fa-b549-f387df03edbb None admin] RBAC: Authorizing `identity:get_role(role_id=03e4cfc753754643ae08fa909647cfe3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:52.971100 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4609b-a4c9-49fa-b549-f387df03edbb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:52.973052 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 555/1120] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:49:52 2022] GET /identity/v3/roles/03e4cfc753754643ae08fa909647cfe3 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:52.979024 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e0df9-e96f-43ce-a582-5b9251bee8d2 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:53.037384 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:53.073841 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e0df9-e96f-43ce-a582-5b9251bee8d2 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:53.106586 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c59cbb-9982-49e9-aee4-4e3f20be8cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:53.107435 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c59cbb-9982-49e9-aee4-4e3f20be8cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:53.107711 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c59cbb-9982-49e9-aee4-4e3f20be8cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:53.107977 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c59cbb-9982-49e9-aee4-4e3f20be8cd0 None admin] PATH_INFO: `/v3/users/1f4237db19f840f6b5228a11348be6b8` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:53.109159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c59cbb-9982-49e9-aee4-4e3f20be8cd0 None admin] RBAC: Authorizing `identity:get_user(user_id=1f4237db19f840f6b5228a11348be6b8)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:53.110594 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c59cbb-9982-49e9-aee4-4e3f20be8cd0 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:53.112181 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 566/1121] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:49:52 2022] GET /identity/v3/users/1f4237db19f840f6b5228a11348be6b8 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:53.117919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4609b-a4c9-49fa-b549-f387df03edbb None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:53.214689 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4609b-a4c9-49fa-b549-f387df03edbb None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:53.247172 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769c524-3fcf-427b-9750-c9ab3f9cde97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:53.248020 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769c524-3fcf-427b-9750-c9ab3f9cde97 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:53.248327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769c524-3fcf-427b-9750-c9ab3f9cde97 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:53.248592 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769c524-3fcf-427b-9750-c9ab3f9cde97 None admin] PATH_INFO: `/v3/projects/ce5366140a8d43b3bce1dc77ef8e6859` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:53.249850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d769c524-3fcf-427b-9750-c9ab3f9cde97 None admin] RBAC: Authorizing `identity:get_project(project_id=ce5366140a8d43b3bce1dc77ef8e6859)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:53.251269 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d769c524-3fcf-427b-9750-c9ab3f9cde97 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:53.253500 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 556/1122] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:49:53 2022] GET /identity/v3/projects/ce5366140a8d43b3bce1dc77ef8e6859 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:53.259325 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c59cbb-9982-49e9-aee4-4e3f20be8cd0 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:53.354571 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c59cbb-9982-49e9-aee4-4e3f20be8cd0 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:53.381516 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:53.387017 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b7a7f7-5d16-42e7-84b1-aa87337b4155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:53.387877 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7a7f7-5d16-42e7-84b1-aa87337b4155 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:53.388154 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7a7f7-5d16-42e7-84b1-aa87337b4155 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:53.388452 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b7a7f7-5d16-42e7-84b1-aa87337b4155 None admin] PATH_INFO: `/v3/projects/ce5366140a8d43b3bce1dc77ef8e6859/users/1f4237db19f840f6b5228a11348be6b8/roles/03e4cfc753754643ae08fa909647cfe3` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:53.390637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b7a7f7-5d16-42e7-84b1-aa87337b4155 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce5366140a8d43b3bce1dc77ef8e6859, user_id=1f4237db19f840f6b5228a11348be6b8, role_id=03e4cfc753754643ae08fa909647cfe3)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:53.392050 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b7a7f7-5d16-42e7-84b1-aa87337b4155 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:53.470153 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 567/1123] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jun 23 19:49:53 2022] PUT /identity/v3/projects/ce5366140a8d43b3bce1dc77ef8e6859/users/1f4237db19f840f6b5228a11348be6b8/roles/03e4cfc753754643ae08fa909647cfe3 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:49:55.485044 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a48c5-b968-4ab7-9782-f30a663403ea None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:55.485588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a48c5-b968-4ab7-9782-f30a663403ea None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:55.485588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a48c5-b968-4ab7-9782-f30a663403ea None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:55.487034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 557/1124] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:55.492053 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883669c-27d8-4f28-bf87-84a249f21ae5 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:55.492502 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883669c-27d8-4f28-bf87-84a249f21ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:55.492637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883669c-27d8-4f28-bf87-84a249f21ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:55.509549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-5883669c-27d8-4f28-bf87-84a249f21ae5 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:55.510722 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5883669c-27d8-4f28-bf87-84a249f21ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:55.575512 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 568/1125] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:55.599768 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a48c5-b968-4ab7-9782-f30a663403ea None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:55.659558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:55.696397 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a48c5-b968-4ab7-9782-f30a663403ea None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:55.728466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ceec5-4151-408a-884c-f4e059ae8a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:55.729293 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ceec5-4151-408a-884c-f4e059ae8a4e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:55.729570 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ceec5-4151-408a-884c-f4e059ae8a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:55.729844 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ceec5-4151-408a-884c-f4e059ae8a4e None admin] PATH_INFO: `/v3/roles/03e4cfc753754643ae08fa909647cfe3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:55.731271 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ceec5-4151-408a-884c-f4e059ae8a4e None admin] RBAC: Authorizing `identity:get_role(role_id=03e4cfc753754643ae08fa909647cfe3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:55.732646 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ceec5-4151-408a-884c-f4e059ae8a4e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:55.733979 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 558/1126] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:49:55 2022] GET /identity/v3/roles/03e4cfc753754643ae08fa909647cfe3 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:55.740351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883669c-27d8-4f28-bf87-84a249f21ae5 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:55.834754 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883669c-27d8-4f28-bf87-84a249f21ae5 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:55.853740 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:55.867381 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8cfc00-188d-4b35-a56c-6724ffebf03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:55.868241 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cfc00-188d-4b35-a56c-6724ffebf03c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:55.868556 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cfc00-188d-4b35-a56c-6724ffebf03c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:55.868830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cfc00-188d-4b35-a56c-6724ffebf03c None admin] PATH_INFO: `/v3/users/1f4237db19f840f6b5228a11348be6b8` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:55.870014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8cfc00-188d-4b35-a56c-6724ffebf03c None admin] RBAC: Authorizing `identity:get_user(user_id=1f4237db19f840f6b5228a11348be6b8)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:55.871459 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8cfc00-188d-4b35-a56c-6724ffebf03c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:55.873086 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 569/1127] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:49:55 2022] GET /identity/v3/users/1f4237db19f840f6b5228a11348be6b8 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:55.879544 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ceec5-4151-408a-884c-f4e059ae8a4e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:55.978734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ceec5-4151-408a-884c-f4e059ae8a4e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:56.009229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:56.013969 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab6cf95-1ef8-48cf-badc-9047e16efad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:56.014624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6cf95-1ef8-48cf-badc-9047e16efad2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:56.014909 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6cf95-1ef8-48cf-badc-9047e16efad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:56.015185 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6cf95-1ef8-48cf-badc-9047e16efad2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:56.082853 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab6cf95-1ef8-48cf-badc-9047e16efad2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:56.084338 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab6cf95-1ef8-48cf-badc-9047e16efad2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:56.180198 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 559/1128] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:49:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 19:49:56.186302 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8cfc00-188d-4b35-a56c-6724ffebf03c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:56.289100 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8cfc00-188d-4b35-a56c-6724ffebf03c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:56.323601 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e039b36-3d0f-4909-93d2-0ec3706771bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:56.324476 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e039b36-3d0f-4909-93d2-0ec3706771bc None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:56.324762 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e039b36-3d0f-4909-93d2-0ec3706771bc None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:56.325037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e039b36-3d0f-4909-93d2-0ec3706771bc None admin] PATH_INFO: `/v3/projects/ce5366140a8d43b3bce1dc77ef8e6859` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:56.326214 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e039b36-3d0f-4909-93d2-0ec3706771bc None admin] RBAC: Authorizing `identity:get_project(project_id=ce5366140a8d43b3bce1dc77ef8e6859)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:56.327660 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e039b36-3d0f-4909-93d2-0ec3706771bc None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:56.329380 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 570/1129] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:49:56 2022] GET /identity/v3/projects/ce5366140a8d43b3bce1dc77ef8e6859 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:56.335457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab6cf95-1ef8-48cf-badc-9047e16efad2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:56.365412 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:56.430815 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab6cf95-1ef8-48cf-badc-9047e16efad2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:56.462883 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980774a-14d9-4efc-9b6f-f543ae8781f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:56.463506 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980774a-14d9-4efc-9b6f-f543ae8781f3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:56.463781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980774a-14d9-4efc-9b6f-f543ae8781f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:56.464049 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980774a-14d9-4efc-9b6f-f543ae8781f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:56.464890 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980774a-14d9-4efc-9b6f-f543ae8781f3 None admin] RBAC: Adding query filter params (role.id=03e4cfc753754643ae08fa909647cfe3, scope.project.id=ce5366140a8d43b3bce1dc77ef8e6859, user.id=1f4237db19f840f6b5228a11348be6b8) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:49:56.465195 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980774a-14d9-4efc-9b6f-f543ae8781f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:56.466618 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980774a-14d9-4efc-9b6f-f543ae8781f3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:56.472799 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 560/1130] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:49:56 2022] GET /identity/v3/role_assignments?user.id=1f4237db19f840f6b5228a11348be6b8&scope.project.id=ce5366140a8d43b3bce1dc77ef8e6859&role.id=03e4cfc753754643ae08fa909647cfe3 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:58.510536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47bfe5-ca33-4aac-872d-63e46f99055f None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:58.511037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47bfe5-ca33-4aac-872d-63e46f99055f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:58.511037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47bfe5-ca33-4aac-872d-63e46f99055f None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:58.512373 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 571/1131] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:49:58.517418 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc0af2-6f55-4747-a545-9621eb2dd3c6 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:58.517868 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc0af2-6f55-4747-a545-9621eb2dd3c6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:58.517964 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc0af2-6f55-4747-a545-9621eb2dd3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:58.536784 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-06cc0af2-6f55-4747-a545-9621eb2dd3c6 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:49:58.538574 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-06cc0af2-6f55-4747-a545-9621eb2dd3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:49:58.605413 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 561/1132] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:49:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:49:58.629793 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47bfe5-ca33-4aac-872d-63e46f99055f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:58.685229 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:58.727683 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47bfe5-ca33-4aac-872d-63e46f99055f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:58.759970 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf611a74-4ee8-4a45-8450-de223bddadc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:58.760819 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf611a74-4ee8-4a45-8450-de223bddadc5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:58.761050 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf611a74-4ee8-4a45-8450-de223bddadc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:58.761319 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf611a74-4ee8-4a45-8450-de223bddadc5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:58.762473 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf611a74-4ee8-4a45-8450-de223bddadc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:58.763903 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf611a74-4ee8-4a45-8450-de223bddadc5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:58.765512 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 572/1133] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:49:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:49:58.771238 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc0af2-6f55-4747-a545-9621eb2dd3c6 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:49:58.797774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:49:58.868059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc0af2-6f55-4747-a545-9621eb2dd3c6 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:49:58.900467 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfbee4c-8440-4947-9d70-f43b8f6a8f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:49:58.901110 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbee4c-8440-4947-9d70-f43b8f6a8f8d None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:49:58.901388 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbee4c-8440-4947-9d70-f43b8f6a8f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:49:58.901661 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbee4c-8440-4947-9d70-f43b8f6a8f8d None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:49:58.902353 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfbee4c-8440-4947-9d70-f43b8f6a8f8d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:49:58.903774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfbee4c-8440-4947-9d70-f43b8f6a8f8d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:49:59.077340 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 562/1134] 10.222.0.44 () {60 vars in 1141 bytes} [Thu Jun 23 19:49:58 2022] POST /identity/v3/projects => generated 311 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:50:01.118160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2693835-1fab-435d-b3da-8475933d616e None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:01.118687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2693835-1fab-435d-b3da-8475933d616e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:01.118687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2693835-1fab-435d-b3da-8475933d616e None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:01.120047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 573/1135] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:50:01.125129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88552d23-1540-4b9d-b208-ebc2613c630b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:01.125543 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88552d23-1540-4b9d-b208-ebc2613c630b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:01.125689 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88552d23-1540-4b9d-b208-ebc2613c630b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:01.142422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-88552d23-1540-4b9d-b208-ebc2613c630b None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:50:01.143627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-88552d23-1540-4b9d-b208-ebc2613c630b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:50:01.207115 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 563/1136] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:50:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:50:01.231307 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2693835-1fab-435d-b3da-8475933d616e None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:01.336260 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2693835-1fab-435d-b3da-8475933d616e None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:01.358472 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:01.374142 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458a2b66-d697-419d-8741-359424e19c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:01.374142 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a2b66-d697-419d-8741-359424e19c76 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:01.374142 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a2b66-d697-419d-8741-359424e19c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:01.374142 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458a2b66-d697-419d-8741-359424e19c76 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:01.374510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a2b66-d697-419d-8741-359424e19c76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:01.374510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458a2b66-d697-419d-8741-359424e19c76 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:01.374710 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 574/1137] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:50:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:50:01.381171 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88552d23-1540-4b9d-b208-ebc2613c630b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:01.391088 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:01.476951 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88552d23-1540-4b9d-b208-ebc2613c630b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:01.510579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307e310-badf-41ee-a82c-3390434ef5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:01.511241 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307e310-badf-41ee-a82c-3390434ef5a3 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:01.511532 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307e310-badf-41ee-a82c-3390434ef5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:01.511805 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307e310-badf-41ee-a82c-3390434ef5a3 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:01.512550 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307e310-badf-41ee-a82c-3390434ef5a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:01.513990 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307e310-badf-41ee-a82c-3390434ef5a3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:01.635683 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 564/1138] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jun 23 19:50:01 2022] POST /identity/v3/users => generated 286 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:50:03.671309 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c495e57-784a-4999-9764-e3c22ffa2341 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:03.671838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c495e57-784a-4999-9764-e3c22ffa2341 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:03.671838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c495e57-784a-4999-9764-e3c22ffa2341 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:03.673234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 575/1139] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:50:03.678262 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d33eb-d5f7-4cd7-8651-8e8c602def10 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:03.678681 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d33eb-d5f7-4cd7-8651-8e8c602def10 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:03.678824 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d33eb-d5f7-4cd7-8651-8e8c602def10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:03.696148 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c37d33eb-d5f7-4cd7-8651-8e8c602def10 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:50:03.697403 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c37d33eb-d5f7-4cd7-8651-8e8c602def10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:50:03.753681 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:03.761764 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 565/1140] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:50:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:50:03.785124 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c495e57-784a-4999-9764-e3c22ffa2341 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:03.882337 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c495e57-784a-4999-9764-e3c22ffa2341 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:03.914454 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c262d-d7ed-43a3-9853-6d439b9f5999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:03.915222 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c262d-d7ed-43a3-9853-6d439b9f5999 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:03.915502 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c262d-d7ed-43a3-9853-6d439b9f5999 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:03.915775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c262d-d7ed-43a3-9853-6d439b9f5999 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:03.925504 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4c262d-d7ed-43a3-9853-6d439b9f5999 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:50:03.932935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-df4c262d-d7ed-43a3-9853-6d439b9f5999 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:50:03.932935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:50:03.932935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:50:03.932935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:50:03.932935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:50:03.932935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:50:03.932935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:50:03.932935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:50:03.932935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:50:03.932935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:50:03.932935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:50:03.932935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:50:03.932935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:50:03.932935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:50:03.932935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:50:03.932935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:50:03.934042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:50:03.934042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:50:03.934042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:50:03.934042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:50:03.934042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:50:03.934042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:50:03.934042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:50:03.934042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:50:03.934042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:50:03.934042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:50:03.934042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:50:03.934042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:50:03.934042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:50:03.934042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:50:03.934042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:50:03.934042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:50:03.934935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:50:03.934935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:50:03.934935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:50:03.934935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:50:03.934935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:50:03.934935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:50:03.934935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:50:03.934935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:50:03.934935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:50:03.934935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:50:03.934935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:50:03.934935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:50:03.934935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 576/1141] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:50:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:50:03.940677 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d33eb-d5f7-4cd7-8651-8e8c602def10 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:04.035694 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d33eb-d5f7-4cd7-8651-8e8c602def10 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:04.068183 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c78fa-9df7-4a97-a76c-985c1758b72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:04.068859 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c78fa-9df7-4a97-a76c-985c1758b72d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:04.069139 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c78fa-9df7-4a97-a76c-985c1758b72d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:04.069414 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c78fa-9df7-4a97-a76c-985c1758b72d None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:04.070154 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c78fa-9df7-4a97-a76c-985c1758b72d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:50:04.070461 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c78fa-9df7-4a97-a76c-985c1758b72d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:04.071846 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c78fa-9df7-4a97-a76c-985c1758b72d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:04.079576 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 566/1142] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:50:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:50:04.085198 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c262d-d7ed-43a3-9853-6d439b9f5999 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:04.135961 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:04.181032 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4c262d-d7ed-43a3-9853-6d439b9f5999 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:04.213324 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc9a04-68d3-49b4-bb61-1bb615532374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:04.214169 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc9a04-68d3-49b4-bb61-1bb615532374 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:04.214450 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc9a04-68d3-49b4-bb61-1bb615532374 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:04.214724 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc9a04-68d3-49b4-bb61-1bb615532374 None admin] PATH_INFO: `/v3/users/104e5b6126374ae5bf8909ff3bd626b2` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:04.227333 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc9a04-68d3-49b4-bb61-1bb615532374 None admin] RBAC: Authorizing `identity:get_user(user_id=104e5b6126374ae5bf8909ff3bd626b2)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:04.228820 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc9a04-68d3-49b4-bb61-1bb615532374 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:04.230459 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 577/1143] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:50:04 2022] GET /identity/v3/users/104e5b6126374ae5bf8909ff3bd626b2 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:50:04.236160 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c78fa-9df7-4a97-a76c-985c1758b72d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:04.290659 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:04.333406 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c78fa-9df7-4a97-a76c-985c1758b72d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:04.365226 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8949e8-55b2-4577-a27a-1f1a73a1e965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:04.365852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8949e8-55b2-4577-a27a-1f1a73a1e965 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:04.366134 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8949e8-55b2-4577-a27a-1f1a73a1e965 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:04.366406 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8949e8-55b2-4577-a27a-1f1a73a1e965 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:04.430146 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8949e8-55b2-4577-a27a-1f1a73a1e965 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:04.431626 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8949e8-55b2-4577-a27a-1f1a73a1e965 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:04.483467 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:04.525633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 567/1144] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:50:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 19:50:04.531405 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc9a04-68d3-49b4-bb61-1bb615532374 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:04.647925 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc9a04-68d3-49b4-bb61-1bb615532374 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:04.658824 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:04.681436 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6096006a-dde2-4aa8-8142-7f6d0a1e38ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:04.682335 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096006a-dde2-4aa8-8142-7f6d0a1e38ec None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:04.682567 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096006a-dde2-4aa8-8142-7f6d0a1e38ec None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:04.682836 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096006a-dde2-4aa8-8142-7f6d0a1e38ec None admin] PATH_INFO: `/v3/projects/d2fab3f12d834ea48d01f5329e8e55ce` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:04.684001 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6096006a-dde2-4aa8-8142-7f6d0a1e38ec None admin] RBAC: Authorizing `identity:get_project(project_id=d2fab3f12d834ea48d01f5329e8e55ce)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:04.685472 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6096006a-dde2-4aa8-8142-7f6d0a1e38ec None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:04.688001 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 578/1145] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:50:04 2022] GET /identity/v3/projects/d2fab3f12d834ea48d01f5329e8e55ce => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:50:04.693405 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8949e8-55b2-4577-a27a-1f1a73a1e965 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:04.787609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8949e8-55b2-4577-a27a-1f1a73a1e965 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:04.820138 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2efa77-c1a0-404b-aa27-fb3319aae097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:04.820802 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2efa77-c1a0-404b-aa27-fb3319aae097 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:04.821085 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2efa77-c1a0-404b-aa27-fb3319aae097 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:04.821359 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2efa77-c1a0-404b-aa27-fb3319aae097 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:04.822172 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2efa77-c1a0-404b-aa27-fb3319aae097 None admin] RBAC: Adding query filter params (role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb, scope.project.id=d2fab3f12d834ea48d01f5329e8e55ce, user.id=104e5b6126374ae5bf8909ff3bd626b2) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:50:04.822480 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2efa77-c1a0-404b-aa27-fb3319aae097 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:04.823890 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2efa77-c1a0-404b-aa27-fb3319aae097 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:04.829136 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 568/1146] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:50:04 2022] GET /identity/v3/role_assignments?user.id=104e5b6126374ae5bf8909ff3bd626b2&scope.project.id=d2fab3f12d834ea48d01f5329e8e55ce&role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:50:06.825920 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136b77e-711d-4363-8e98-9fd7d3f537c5 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:06.826479 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136b77e-711d-4363-8e98-9fd7d3f537c5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:06.826479 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136b77e-711d-4363-8e98-9fd7d3f537c5 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:06.827817 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 579/1147] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:50:06.832978 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73dde41-7978-4657-b1ed-b7bd509532ef None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:06.833344 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73dde41-7978-4657-b1ed-b7bd509532ef None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:06.833531 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73dde41-7978-4657-b1ed-b7bd509532ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:06.850250 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f73dde41-7978-4657-b1ed-b7bd509532ef None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:50:06.851409 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f73dde41-7978-4657-b1ed-b7bd509532ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:50:06.864865 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:06.915477 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 569/1148] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:50:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:50:06.937560 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1136b77e-711d-4363-8e98-9fd7d3f537c5 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:07.035387 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1136b77e-711d-4363-8e98-9fd7d3f537c5 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:07.067827 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a0d41-7897-4547-b2aa-506d272c4050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:07.068620 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a0d41-7897-4547-b2aa-506d272c4050 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:07.068899 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a0d41-7897-4547-b2aa-506d272c4050 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:07.069168 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a0d41-7897-4547-b2aa-506d272c4050 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:07.078847 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a0d41-7897-4547-b2aa-506d272c4050 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:50:07.086768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-180a0d41-7897-4547-b2aa-506d272c4050 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:50:07.086768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:50:07.086768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:50:07.086768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:50:07.086768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:50:07.086768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:50:07.086768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:50:07.086768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:50:07.086768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:50:07.086768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:50:07.086768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:50:07.086768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:50:07.086768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:50:07.086768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:50:07.086768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:50:07.086768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:50:07.087890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:50:07.087890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:50:07.087890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:50:07.087890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:50:07.087890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:50:07.087890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:50:07.087890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:50:07.087890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:50:07.087890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:50:07.087890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:50:07.087890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:50:07.087890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:50:07.087890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:50:07.087890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:50:07.087890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:50:07.087890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:50:07.088802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:50:07.088802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:50:07.088802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:50:07.088802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:50:07.088802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:50:07.088802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:50:07.088802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:50:07.088802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:50:07.088802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:50:07.088802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:50:07.088802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:50:07.088802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:50:07.088802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 580/1149] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:50:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:50:07.093825 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73dde41-7978-4657-b1ed-b7bd509532ef None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:07.188822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73dde41-7978-4657-b1ed-b7bd509532ef None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:07.221246 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03bcba5-1736-4d5a-864a-b823989ea88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:07.221888 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03bcba5-1736-4d5a-864a-b823989ea88e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:07.222165 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03bcba5-1736-4d5a-864a-b823989ea88e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:07.222434 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03bcba5-1736-4d5a-864a-b823989ea88e None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:07.223163 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03bcba5-1736-4d5a-864a-b823989ea88e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:50:07.223492 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03bcba5-1736-4d5a-864a-b823989ea88e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:07.224894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03bcba5-1736-4d5a-864a-b823989ea88e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:07.232852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 570/1150] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:50:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:50:07.238976 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a0d41-7897-4547-b2aa-506d272c4050 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:07.277568 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:07.335287 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a0d41-7897-4547-b2aa-506d272c4050 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:07.367777 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a226332-e852-4bc9-85df-ae83898d35f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:07.368660 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a226332-e852-4bc9-85df-ae83898d35f6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:07.368939 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a226332-e852-4bc9-85df-ae83898d35f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:07.369206 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a226332-e852-4bc9-85df-ae83898d35f6 None admin] PATH_INFO: `/v3/users/104e5b6126374ae5bf8909ff3bd626b2` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:07.370361 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a226332-e852-4bc9-85df-ae83898d35f6 None admin] RBAC: Authorizing `identity:get_user(user_id=104e5b6126374ae5bf8909ff3bd626b2)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:07.371798 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a226332-e852-4bc9-85df-ae83898d35f6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:07.373413 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 581/1151] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:50:07 2022] GET /identity/v3/users/104e5b6126374ae5bf8909ff3bd626b2 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:50:07.379602 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03bcba5-1736-4d5a-864a-b823989ea88e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:07.394238 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:07.487635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03bcba5-1736-4d5a-864a-b823989ea88e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:07.531739 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b4dfb-5e2f-48cd-bf96-c50aa039340e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:07.532980 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b4dfb-5e2f-48cd-bf96-c50aa039340e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:07.533949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b4dfb-5e2f-48cd-bf96-c50aa039340e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:07.534367 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b4dfb-5e2f-48cd-bf96-c50aa039340e None admin] PATH_INFO: `/v3/projects/d2fab3f12d834ea48d01f5329e8e55ce` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:07.535983 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b4dfb-5e2f-48cd-bf96-c50aa039340e None admin] RBAC: Authorizing `identity:get_project(project_id=d2fab3f12d834ea48d01f5329e8e55ce)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:07.537965 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b4dfb-5e2f-48cd-bf96-c50aa039340e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:07.540360 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 571/1152] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:50:07 2022] GET /identity/v3/projects/d2fab3f12d834ea48d01f5329e8e55ce => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:50:07.546539 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a226332-e852-4bc9-85df-ae83898d35f6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:07.679461 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a226332-e852-4bc9-85df-ae83898d35f6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:07.689840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:07.726452 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce18737d-5bda-4f5e-a80e-ff426324363a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:07.727696 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18737d-5bda-4f5e-a80e-ff426324363a None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:07.728119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18737d-5bda-4f5e-a80e-ff426324363a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:07.728535 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce18737d-5bda-4f5e-a80e-ff426324363a None admin] PATH_INFO: `/v3/projects/d2fab3f12d834ea48d01f5329e8e55ce/users/104e5b6126374ae5bf8909ff3bd626b2/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:07.731202 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce18737d-5bda-4f5e-a80e-ff426324363a None admin] RBAC: Authorizing `identity:create_grant(project_id=d2fab3f12d834ea48d01f5329e8e55ce, user_id=104e5b6126374ae5bf8909ff3bd626b2, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:07.733355 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce18737d-5bda-4f5e-a80e-ff426324363a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:07.831475 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 582/1153] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jun 23 19:50:07 2022] PUT /identity/v3/projects/d2fab3f12d834ea48d01f5329e8e55ce/users/104e5b6126374ae5bf8909ff3bd626b2/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:50:09.858530 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65ef0a-517f-4023-b500-b7a4d08b1205 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:09.859114 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65ef0a-517f-4023-b500-b7a4d08b1205 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:09.859114 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65ef0a-517f-4023-b500-b7a4d08b1205 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:09.860563 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 572/1154] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:50:09.866494 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05545b-2929-48b7-ad93-b1144afa30a3 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:09.866914 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05545b-2929-48b7-ad93-b1144afa30a3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:09.867051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05545b-2929-48b7-ad93-b1144afa30a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:09.883420 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ef05545b-2929-48b7-ad93-b1144afa30a3 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:50:09.884659 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ef05545b-2929-48b7-ad93-b1144afa30a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:50:09.949673 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 583/1155] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:50:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:50:09.974658 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da65ef0a-517f-4023-b500-b7a4d08b1205 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:10.056788 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:10.076513 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da65ef0a-517f-4023-b500-b7a4d08b1205 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:10.110551 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90fb420-7c42-4030-bc33-1901a4eeab4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:10.111810 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90fb420-7c42-4030-bc33-1901a4eeab4e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:10.112094 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90fb420-7c42-4030-bc33-1901a4eeab4e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:10.112416 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90fb420-7c42-4030-bc33-1901a4eeab4e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:10.122529 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90fb420-7c42-4030-bc33-1901a4eeab4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:50:10.130936 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-c90fb420-7c42-4030-bc33-1901a4eeab4e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:50:10.130936 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:50:10.130936 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:50:10.130936 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:50:10.130936 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:50:10.130936 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:50:10.130936 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:50:10.130936 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:50:10.130936 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:50:10.130936 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:50:10.130936 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:50:10.130936 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:50:10.130936 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:50:10.130936 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:50:10.130936 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:50:10.130936 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:50:10.132126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:50:10.132126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:50:10.132126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:50:10.132126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:50:10.132126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:50:10.132126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:50:10.132126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:50:10.132126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:50:10.132126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:50:10.132126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:50:10.132126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:50:10.132126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:50:10.132126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:50:10.132126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:50:10.132126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:50:10.132126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:50:10.133229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:50:10.133229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:50:10.133229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:50:10.133229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:50:10.133229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:50:10.133229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:50:10.133229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:50:10.133229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:50:10.133229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:50:10.133229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:50:10.133229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:50:10.133229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:50:10.133229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 573/1156] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:50:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:50:10.138324 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05545b-2929-48b7-ad93-b1144afa30a3 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:10.228737 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:10.234489 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05545b-2929-48b7-ad93-b1144afa30a3 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:10.268588 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4da19-2a38-4537-b41a-86bc1c069632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:10.269254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4da19-2a38-4537-b41a-86bc1c069632 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:10.269540 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4da19-2a38-4537-b41a-86bc1c069632 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:10.269817 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4da19-2a38-4537-b41a-86bc1c069632 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:10.270559 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f4da19-2a38-4537-b41a-86bc1c069632 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:50:10.270870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f4da19-2a38-4537-b41a-86bc1c069632 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:10.272256 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f4da19-2a38-4537-b41a-86bc1c069632 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:10.280033 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 584/1157] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:50:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:50:10.286291 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90fb420-7c42-4030-bc33-1901a4eeab4e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:10.384551 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90fb420-7c42-4030-bc33-1901a4eeab4e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:10.418541 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23157809-7d04-4c4f-a102-a62c46e70308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:10.419434 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23157809-7d04-4c4f-a102-a62c46e70308 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:10.419729 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23157809-7d04-4c4f-a102-a62c46e70308 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:10.420012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23157809-7d04-4c4f-a102-a62c46e70308 None admin] PATH_INFO: `/v3/users/104e5b6126374ae5bf8909ff3bd626b2` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:10.421243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23157809-7d04-4c4f-a102-a62c46e70308 None admin] RBAC: Authorizing `identity:get_user(user_id=104e5b6126374ae5bf8909ff3bd626b2)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:10.422768 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23157809-7d04-4c4f-a102-a62c46e70308 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:10.424520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 574/1158] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:50:10 2022] GET /identity/v3/users/104e5b6126374ae5bf8909ff3bd626b2 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:50:10.430260 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4da19-2a38-4537-b41a-86bc1c069632 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:10.527967 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4da19-2a38-4537-b41a-86bc1c069632 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:10.559679 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd0dc96-8cd1-4c5c-b8c3-ad60d6ede836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:10.560328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0dc96-8cd1-4c5c-b8c3-ad60d6ede836 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:10.560673 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0dc96-8cd1-4c5c-b8c3-ad60d6ede836 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:10.560947 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0dc96-8cd1-4c5c-b8c3-ad60d6ede836 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:10.587487 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:10.625550 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd0dc96-8cd1-4c5c-b8c3-ad60d6ede836 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:10.627608 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd0dc96-8cd1-4c5c-b8c3-ad60d6ede836 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:10.734415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 585/1159] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:50:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 19:50:10.740349 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23157809-7d04-4c4f-a102-a62c46e70308 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:10.751420 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:10.838009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23157809-7d04-4c4f-a102-a62c46e70308 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:10.870785 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0557468-9991-4390-998c-1561ddd8d75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:10.871633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0557468-9991-4390-998c-1561ddd8d75d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:10.871915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0557468-9991-4390-998c-1561ddd8d75d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:10.872188 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0557468-9991-4390-998c-1561ddd8d75d None admin] PATH_INFO: `/v3/projects/d2fab3f12d834ea48d01f5329e8e55ce` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:10.873412 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0557468-9991-4390-998c-1561ddd8d75d None admin] RBAC: Authorizing `identity:get_project(project_id=d2fab3f12d834ea48d01f5329e8e55ce)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:10.874852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0557468-9991-4390-998c-1561ddd8d75d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:10.876580 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 575/1160] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:50:10 2022] GET /identity/v3/projects/d2fab3f12d834ea48d01f5329e8e55ce => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:50:10.883526 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd0dc96-8cd1-4c5c-b8c3-ad60d6ede836 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:10.940535 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:10.978709 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd0dc96-8cd1-4c5c-b8c3-ad60d6ede836 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:11.012015 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9396e-2fcd-4588-a56a-eeec5696cff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:11.012783 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9396e-2fcd-4588-a56a-eeec5696cff7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:11.013066 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9396e-2fcd-4588-a56a-eeec5696cff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:11.013335 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9396e-2fcd-4588-a56a-eeec5696cff7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:11.014158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9396e-2fcd-4588-a56a-eeec5696cff7 None admin] RBAC: Adding query filter params (role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb, scope.project.id=d2fab3f12d834ea48d01f5329e8e55ce, user.id=104e5b6126374ae5bf8909ff3bd626b2) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:50:11.014464 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9396e-2fcd-4588-a56a-eeec5696cff7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:11.015900 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9396e-2fcd-4588-a56a-eeec5696cff7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:11.021756 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 586/1161] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:50:10 2022] GET /identity/v3/role_assignments?user.id=104e5b6126374ae5bf8909ff3bd626b2&scope.project.id=d2fab3f12d834ea48d01f5329e8e55ce&role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:50:13.066608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf3f41-b2f0-4504-a188-0b1b55edc430 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:13.067137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf3f41-b2f0-4504-a188-0b1b55edc430 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:13.067256 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf3f41-b2f0-4504-a188-0b1b55edc430 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:13.068668 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 576/1162] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:50:13.073745 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730bece-a60a-4419-abe8-47d5f6ad4299 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:13.074179 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730bece-a60a-4419-abe8-47d5f6ad4299 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:13.074279 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730bece-a60a-4419-abe8-47d5f6ad4299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:13.091851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9730bece-a60a-4419-abe8-47d5f6ad4299 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:50:13.093078 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9730bece-a60a-4419-abe8-47d5f6ad4299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:50:13.156859 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 587/1163] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:50:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:50:13.179377 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf3f41-b2f0-4504-a188-0b1b55edc430 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:13.255853 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:13.278843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf3f41-b2f0-4504-a188-0b1b55edc430 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:13.312050 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfe47f-452e-47bb-804e-a7cb6b2e47d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:13.313041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfe47f-452e-47bb-804e-a7cb6b2e47d6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:13.313323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfe47f-452e-47bb-804e-a7cb6b2e47d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:13.313594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfe47f-452e-47bb-804e-a7cb6b2e47d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:13.377301 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfe47f-452e-47bb-804e-a7cb6b2e47d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:13.378692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfe47f-452e-47bb-804e-a7cb6b2e47d6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:13.449708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:13.472014 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 577/1164] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:50:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 19:50:13.478030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730bece-a60a-4419-abe8-47d5f6ad4299 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:13.527800 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:13.572924 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730bece-a60a-4419-abe8-47d5f6ad4299 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:13.605616 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5600b0ed-6819-4b4f-a87c-785c58bc2d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:13.606401 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600b0ed-6819-4b4f-a87c-785c58bc2d02 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:13.606680 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600b0ed-6819-4b4f-a87c-785c58bc2d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:13.606952 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5600b0ed-6819-4b4f-a87c-785c58bc2d02 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:13.613510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600b0ed-6819-4b4f-a87c-785c58bc2d02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:13.614958 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5600b0ed-6819-4b4f-a87c-785c58bc2d02 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:13.629230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-5600b0ed-6819-4b4f-a87c-785c58bc2d02 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 23 19:50:13.629230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:50:13.629230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 23 19:50:13.629230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 23 19:50:13.629230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:50:13.629230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:50:13.629230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:50:13.629230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:50:13.629230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 23 19:50:13.629230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:50:13.629230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:50:13.629230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:50:13.629230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:50:13.629230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:50:13.629230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:50:13.629230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:50:13.629230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:50:13.630430 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:50:13.630430 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:50:13.630430 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:50:13.630430 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:50:13.630430 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:50:13.630430 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:50:13.630430 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:50:13.630430 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:50:13.630430 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:50:13.630430 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:50:13.630430 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:50:13.630430 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:50:13.630430 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:50:13.630430 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:50:13.630430 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:50:13.630430 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:50:13.631636 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:50:13.631636 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:50:13.631636 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:50:13.631636 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:50:13.631636 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:50:13.631636 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:50:13.631636 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:50:13.631636 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:50:13.631636 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:50:13.631636 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:50:13.631636 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 23 19:50:13.631636 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:50:13.631636 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 23 19:50:13.631636 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:50:13.631636 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 588/1165] 10.222.0.44 () {56 vars in 1117 bytes} [Thu Jun 23 19:50:13 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:50:13.636126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfe47f-452e-47bb-804e-a7cb6b2e47d6 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:13.732502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfe47f-452e-47bb-804e-a7cb6b2e47d6 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:13.765305 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e90a02-5507-42de-aa44-e43b7ddb44a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:13.765946 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e90a02-5507-42de-aa44-e43b7ddb44a1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:13.766228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e90a02-5507-42de-aa44-e43b7ddb44a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:13.766503 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e90a02-5507-42de-aa44-e43b7ddb44a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:13.767275 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e90a02-5507-42de-aa44-e43b7ddb44a1 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:50:13.768098 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e90a02-5507-42de-aa44-e43b7ddb44a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:13.769486 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e90a02-5507-42de-aa44-e43b7ddb44a1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:13.776563 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 578/1166] 10.222.0.44 () {56 vars in 1115 bytes} [Thu Jun 23 19:50:13 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:50:15.747975 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36b048-c60e-4db1-8106-775d8cb57690 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:15.748549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36b048-c60e-4db1-8106-775d8cb57690 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:15.748549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36b048-c60e-4db1-8106-775d8cb57690 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:15.749914 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 589/1167] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:50:15.755191 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d252af9-af7c-48dd-8ee2-8a88b57d92e0 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:15.755626 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d252af9-af7c-48dd-8ee2-8a88b57d92e0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:15.755745 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d252af9-af7c-48dd-8ee2-8a88b57d92e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:15.771788 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9d252af9-af7c-48dd-8ee2-8a88b57d92e0 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:50:15.772983 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9d252af9-af7c-48dd-8ee2-8a88b57d92e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:50:15.805666 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:15.836090 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 579/1168] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:50:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:50:15.859568 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36b048-c60e-4db1-8106-775d8cb57690 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:15.957650 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36b048-c60e-4db1-8106-775d8cb57690 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:15.974508 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:16.002149 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db0ac8a-8e56-4735-a2ea-923ae9d4450a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:16.003162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0ac8a-8e56-4735-a2ea-923ae9d4450a None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:16.003643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0ac8a-8e56-4735-a2ea-923ae9d4450a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:16.004346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0ac8a-8e56-4735-a2ea-923ae9d4450a None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:16.005506 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db0ac8a-8e56-4735-a2ea-923ae9d4450a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:16.007693 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db0ac8a-8e56-4735-a2ea-923ae9d4450a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:16.077053 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 590/1169] 10.222.0.44 () {60 vars in 1138 bytes} [Thu Jun 23 19:50:15 2022] POST /identity/v3/domains => generated 267 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:50:18.132764 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d61d5-27a1-43b9-a005-49e0ccb934a6 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:18.133295 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d61d5-27a1-43b9-a005-49e0ccb934a6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:18.133295 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d61d5-27a1-43b9-a005-49e0ccb934a6 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:18.134670 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 580/1170] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:50:18.139855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49336809-c359-4ae7-a3b9-467280254d31 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:18.140217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49336809-c359-4ae7-a3b9-467280254d31 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:18.140452 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49336809-c359-4ae7-a3b9-467280254d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:18.157208 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-49336809-c359-4ae7-a3b9-467280254d31 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:50:18.158413 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-49336809-c359-4ae7-a3b9-467280254d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:50:18.223259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 591/1171] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:50:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:50:18.246601 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d61d5-27a1-43b9-a005-49e0ccb934a6 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:18.342853 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d61d5-27a1-43b9-a005-49e0ccb934a6 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:18.374798 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac91b9-cbd5-4069-b5df-68017f5be1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:18.375633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac91b9-cbd5-4069-b5df-68017f5be1af None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:18.375906 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac91b9-cbd5-4069-b5df-68017f5be1af None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:18.376172 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac91b9-cbd5-4069-b5df-68017f5be1af None admin] PATH_INFO: `/v3/domains/158c404732df41e9992dacb3feda8dac` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:18.386652 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbac91b9-cbd5-4069-b5df-68017f5be1af None admin] RBAC: Authorizing `identity:get_domain(domain_id=158c404732df41e9992dacb3feda8dac)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:18.388036 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbac91b9-cbd5-4069-b5df-68017f5be1af None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:18.389658 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 581/1172] 10.222.0.44 () {56 vars in 1183 bytes} [Thu Jun 23 19:50:18 2022] GET /identity/v3/domains/158c404732df41e9992dacb3feda8dac => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:50:18.395291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49336809-c359-4ae7-a3b9-467280254d31 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:18.489514 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49336809-c359-4ae7-a3b9-467280254d31 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:18.490982 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:18.521852 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f357ec8-c7bd-475a-9436-db3546552d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:18.522521 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f357ec8-c7bd-475a-9436-db3546552d82 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:18.522775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f357ec8-c7bd-475a-9436-db3546552d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:18.523048 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f357ec8-c7bd-475a-9436-db3546552d82 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:18.523751 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f357ec8-c7bd-475a-9436-db3546552d82 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:18.525241 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f357ec8-c7bd-475a-9436-db3546552d82 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:18.659623 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 592/1173] 10.222.0.44 () {60 vars in 1141 bytes} [Thu Jun 23 19:50:18 2022] POST /identity/v3/projects => generated 361 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:50:20.679213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5fe3f2-57d5-4172-9f0c-cf527cbd1109 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:20.679740 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5fe3f2-57d5-4172-9f0c-cf527cbd1109 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:20.679740 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5fe3f2-57d5-4172-9f0c-cf527cbd1109 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:20.681184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 582/1174] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:50:20.686329 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3c7ff-0317-4894-9ec5-da833800f985 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:20.686691 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3c7ff-0317-4894-9ec5-da833800f985 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:20.686882 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3c7ff-0317-4894-9ec5-da833800f985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:20.703850 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-95b3c7ff-0317-4894-9ec5-da833800f985 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:50:20.705074 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-95b3c7ff-0317-4894-9ec5-da833800f985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:50:20.769597 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 593/1175] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:50:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:50:20.793853 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5fe3f2-57d5-4172-9f0c-cf527cbd1109 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:20.824620 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:20.893778 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5fe3f2-57d5-4172-9f0c-cf527cbd1109 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:20.926610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7318ca5b-ec12-419f-9d4d-89d23523defd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:20.927457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7318ca5b-ec12-419f-9d4d-89d23523defd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:20.927739 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7318ca5b-ec12-419f-9d4d-89d23523defd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:20.928010 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7318ca5b-ec12-419f-9d4d-89d23523defd None admin] PATH_INFO: `/v3/domains/158c404732df41e9992dacb3feda8dac` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:20.929202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7318ca5b-ec12-419f-9d4d-89d23523defd None admin] RBAC: Authorizing `identity:get_domain(domain_id=158c404732df41e9992dacb3feda8dac)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:20.930681 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7318ca5b-ec12-419f-9d4d-89d23523defd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:20.932233 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 583/1176] 10.222.0.44 () {56 vars in 1183 bytes} [Thu Jun 23 19:50:20 2022] GET /identity/v3/domains/158c404732df41e9992dacb3feda8dac => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:50:20.938278 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b3c7ff-0317-4894-9ec5-da833800f985 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:21.017776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:21.033374 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b3c7ff-0317-4894-9ec5-da833800f985 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:21.066672 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2a521f-1d72-4686-9a7c-2a3b6e48d334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:21.074623 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a521f-1d72-4686-9a7c-2a3b6e48d334 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:21.074623 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a521f-1d72-4686-9a7c-2a3b6e48d334 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:21.074623 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2a521f-1d72-4686-9a7c-2a3b6e48d334 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:21.074623 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a521f-1d72-4686-9a7c-2a3b6e48d334 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:21.074623 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2a521f-1d72-4686-9a7c-2a3b6e48d334 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:21.178658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 594/1177] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jun 23 19:50:20 2022] POST /identity/v3/users => generated 311 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:50:23.228011 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e06d1-0ed5-4236-8c8a-6cb64758c9bf None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:23.228563 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e06d1-0ed5-4236-8c8a-6cb64758c9bf None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:23.228677 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e06d1-0ed5-4236-8c8a-6cb64758c9bf None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:23.229960 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 584/1178] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:50:23.235315 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d62611-c3e6-4820-98a3-0c5186ba4bbc None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:23.236249 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d62611-c3e6-4820-98a3-0c5186ba4bbc None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:23.236465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d62611-c3e6-4820-98a3-0c5186ba4bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:23.252718 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-98d62611-c3e6-4820-98a3-0c5186ba4bbc None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:50:23.253892 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-98d62611-c3e6-4820-98a3-0c5186ba4bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:50:23.317224 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 595/1179] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:50:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:50:23.340362 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e06d1-0ed5-4236-8c8a-6cb64758c9bf None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:23.442645 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:23.447234 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e06d1-0ed5-4236-8c8a-6cb64758c9bf None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:23.479461 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e73480-1749-4c42-9abf-fd82514892bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:23.480175 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e73480-1749-4c42-9abf-fd82514892bd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:23.480482 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e73480-1749-4c42-9abf-fd82514892bd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:23.480752 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e73480-1749-4c42-9abf-fd82514892bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:23.490926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e73480-1749-4c42-9abf-fd82514892bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:50:23.498335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-16e73480-1749-4c42-9abf-fd82514892bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:50:23.498335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:50:23.498335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:50:23.498335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:50:23.498335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:50:23.498335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:50:23.498335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:50:23.498335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:50:23.498335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:50:23.498335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:50:23.498335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:50:23.498335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:50:23.498335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:50:23.498335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:50:23.498335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:50:23.498335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:50:23.499430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:50:23.499430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:50:23.499430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:50:23.499430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:50:23.499430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:50:23.499430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:50:23.499430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:50:23.499430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:50:23.499430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:50:23.499430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:50:23.499430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:50:23.499430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:50:23.499430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:50:23.499430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:50:23.499430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:50:23.499430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:50:23.500520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:50:23.500520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:50:23.500520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:50:23.500520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:50:23.500520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:50:23.500520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:50:23.500520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:50:23.500520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:50:23.500520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:50:23.500520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:50:23.500520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:50:23.500520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:50:23.500520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 585/1180] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:50:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:50:23.505545 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d62611-c3e6-4820-98a3-0c5186ba4bbc None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:23.571330 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:23.600447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d62611-c3e6-4820-98a3-0c5186ba4bbc None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:23.632253 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15612fe-d05a-4e15-8345-327404e29b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:23.632931 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15612fe-d05a-4e15-8345-327404e29b53 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:23.633212 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15612fe-d05a-4e15-8345-327404e29b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:23.633486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15612fe-d05a-4e15-8345-327404e29b53 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:23.634223 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15612fe-d05a-4e15-8345-327404e29b53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:50:23.634527 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15612fe-d05a-4e15-8345-327404e29b53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:23.635897 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15612fe-d05a-4e15-8345-327404e29b53 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:23.644535 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 596/1181] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:50:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:50:23.650170 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e73480-1749-4c42-9abf-fd82514892bd None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:23.747254 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e73480-1749-4c42-9abf-fd82514892bd None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:23.779148 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e90deb7-05e4-45be-bdb6-2a5ceb546f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:23.780004 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90deb7-05e4-45be-bdb6-2a5ceb546f6f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:23.780286 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90deb7-05e4-45be-bdb6-2a5ceb546f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:23.780588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90deb7-05e4-45be-bdb6-2a5ceb546f6f None admin] PATH_INFO: `/v3/users/90e704c3aad5448faa134f658fcaa932` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:23.793553 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e90deb7-05e4-45be-bdb6-2a5ceb546f6f None admin] RBAC: Authorizing `identity:get_user(user_id=90e704c3aad5448faa134f658fcaa932)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:23.795016 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e90deb7-05e4-45be-bdb6-2a5ceb546f6f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:23.796684 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 586/1182] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:50:23 2022] GET /identity/v3/users/90e704c3aad5448faa134f658fcaa932 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:50:23.802471 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15612fe-d05a-4e15-8345-327404e29b53 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:23.899035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15612fe-d05a-4e15-8345-327404e29b53 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:23.926631 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:23.931335 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26248e8b-2376-4e8f-84d1-2ef35b6f798a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:23.931966 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26248e8b-2376-4e8f-84d1-2ef35b6f798a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:23.932247 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26248e8b-2376-4e8f-84d1-2ef35b6f798a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:23.932549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26248e8b-2376-4e8f-84d1-2ef35b6f798a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:23.998965 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26248e8b-2376-4e8f-84d1-2ef35b6f798a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:24.000388 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26248e8b-2376-4e8f-84d1-2ef35b6f798a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:24.094537 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 597/1183] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:50:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 19:50:24.100923 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e90deb7-05e4-45be-bdb6-2a5ceb546f6f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:24.118247 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:24.197179 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e90deb7-05e4-45be-bdb6-2a5ceb546f6f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:24.229935 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335122a2-b8fe-4482-a9bb-643a84127fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:24.230791 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335122a2-b8fe-4482-a9bb-643a84127fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:24.231071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335122a2-b8fe-4482-a9bb-643a84127fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:24.231343 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335122a2-b8fe-4482-a9bb-643a84127fb2 None admin] PATH_INFO: `/v3/projects/eb60b2edf7e748048eada504bb0901f5` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:24.232544 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335122a2-b8fe-4482-a9bb-643a84127fb2 None admin] RBAC: Authorizing `identity:get_project(project_id=eb60b2edf7e748048eada504bb0901f5)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:24.233967 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335122a2-b8fe-4482-a9bb-643a84127fb2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:24.235693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 587/1184] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:50:24 2022] GET /identity/v3/projects/eb60b2edf7e748048eada504bb0901f5 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:50:24.241615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26248e8b-2376-4e8f-84d1-2ef35b6f798a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:24.271800 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:24.337170 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26248e8b-2376-4e8f-84d1-2ef35b6f798a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:24.369753 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8be0b6-1aee-4a45-9ca8-a71061adda09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:24.370380 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8be0b6-1aee-4a45-9ca8-a71061adda09 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:24.370661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8be0b6-1aee-4a45-9ca8-a71061adda09 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:24.370931 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8be0b6-1aee-4a45-9ca8-a71061adda09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:24.371744 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8be0b6-1aee-4a45-9ca8-a71061adda09 None admin] RBAC: Adding query filter params (role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb, scope.project.id=eb60b2edf7e748048eada504bb0901f5, user.id=90e704c3aad5448faa134f658fcaa932) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:50:24.372049 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8be0b6-1aee-4a45-9ca8-a71061adda09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:24.373493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8be0b6-1aee-4a45-9ca8-a71061adda09 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:24.378733 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 598/1185] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:50:24 2022] GET /identity/v3/role_assignments?user.id=90e704c3aad5448faa134f658fcaa932&scope.project.id=eb60b2edf7e748048eada504bb0901f5&role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:50:26.362126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69aa653-1a75-4a0b-8075-73d6b9b9092c None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:26.362671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69aa653-1a75-4a0b-8075-73d6b9b9092c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:26.362671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69aa653-1a75-4a0b-8075-73d6b9b9092c None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:26.364030 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 588/1186] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:50:26.369344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffa737-96b0-418a-a27c-9af963bbbd7a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:26.369788 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffa737-96b0-418a-a27c-9af963bbbd7a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:26.369882 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ffa737-96b0-418a-a27c-9af963bbbd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:26.386499 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d4ffa737-96b0-418a-a27c-9af963bbbd7a None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:50:26.387664 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d4ffa737-96b0-418a-a27c-9af963bbbd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:50:26.450959 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 599/1187] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:50:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:50:26.473704 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69aa653-1a75-4a0b-8075-73d6b9b9092c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:26.554737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:26.571173 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69aa653-1a75-4a0b-8075-73d6b9b9092c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:26.603788 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c86eebb-82f2-46b7-ac8a-1c1ab5dc2800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:26.604585 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86eebb-82f2-46b7-ac8a-1c1ab5dc2800 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:26.604870 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86eebb-82f2-46b7-ac8a-1c1ab5dc2800 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:26.605145 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86eebb-82f2-46b7-ac8a-1c1ab5dc2800 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:26.615391 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c86eebb-82f2-46b7-ac8a-1c1ab5dc2800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:50:26.622855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-9c86eebb-82f2-46b7-ac8a-1c1ab5dc2800 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:50:26.622855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:50:26.622855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:50:26.622855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:50:26.622855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:50:26.622855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:50:26.622855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:50:26.622855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:50:26.622855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:50:26.622855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:50:26.622855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:50:26.622855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:50:26.622855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:50:26.622855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:50:26.622855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:50:26.622855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:50:26.623934 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:50:26.623934 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:50:26.623934 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:50:26.623934 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:50:26.623934 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:50:26.623934 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:50:26.623934 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:50:26.623934 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:50:26.623934 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:50:26.623934 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:50:26.623934 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:50:26.623934 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:50:26.623934 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:50:26.623934 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:50:26.623934 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:50:26.623934 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:50:26.625280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:50:26.625280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:50:26.625280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:50:26.625280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:50:26.625280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:50:26.625280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:50:26.625280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:50:26.625280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:50:26.625280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:50:26.625280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:50:26.625280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:50:26.625280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:50:26.625280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 589/1188] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:50:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:50:26.630160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ffa737-96b0-418a-a27c-9af963bbbd7a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:26.655853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:26.725667 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ffa737-96b0-418a-a27c-9af963bbbd7a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:26.760986 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381d872-e20b-4809-96f1-00691c953470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:26.761647 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381d872-e20b-4809-96f1-00691c953470 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:26.761929 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381d872-e20b-4809-96f1-00691c953470 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:26.762200 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381d872-e20b-4809-96f1-00691c953470 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:26.762941 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3381d872-e20b-4809-96f1-00691c953470 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:50:26.763246 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3381d872-e20b-4809-96f1-00691c953470 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:26.764654 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3381d872-e20b-4809-96f1-00691c953470 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:26.776729 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 600/1189] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:50:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:50:26.783714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c86eebb-82f2-46b7-ac8a-1c1ab5dc2800 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:26.880021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c86eebb-82f2-46b7-ac8a-1c1ab5dc2800 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:26.912481 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea28719-073d-4878-95e2-e0d6f8f8aa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:26.913333 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea28719-073d-4878-95e2-e0d6f8f8aa9f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:26.913610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea28719-073d-4878-95e2-e0d6f8f8aa9f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:26.913878 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea28719-073d-4878-95e2-e0d6f8f8aa9f None admin] PATH_INFO: `/v3/users/90e704c3aad5448faa134f658fcaa932` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:26.915041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea28719-073d-4878-95e2-e0d6f8f8aa9f None admin] RBAC: Authorizing `identity:get_user(user_id=90e704c3aad5448faa134f658fcaa932)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:26.916496 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea28719-073d-4878-95e2-e0d6f8f8aa9f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:26.918182 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 590/1190] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:50:26 2022] GET /identity/v3/users/90e704c3aad5448faa134f658fcaa932 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:50:26.923721 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381d872-e20b-4809-96f1-00691c953470 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:27.008947 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:27.020979 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381d872-e20b-4809-96f1-00691c953470 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:27.052538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d57228-02fa-41b9-9d13-53c04b347426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:27.053373 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d57228-02fa-41b9-9d13-53c04b347426 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:27.053656 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d57228-02fa-41b9-9d13-53c04b347426 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:27.053922 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d57228-02fa-41b9-9d13-53c04b347426 None admin] PATH_INFO: `/v3/projects/eb60b2edf7e748048eada504bb0901f5` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:27.055067 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d57228-02fa-41b9-9d13-53c04b347426 None admin] RBAC: Authorizing `identity:get_project(project_id=eb60b2edf7e748048eada504bb0901f5)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:27.056535 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d57228-02fa-41b9-9d13-53c04b347426 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:27.058191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 601/1191] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:50:26 2022] GET /identity/v3/projects/eb60b2edf7e748048eada504bb0901f5 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:50:27.064813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea28719-073d-4878-95e2-e0d6f8f8aa9f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:27.079867 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:27.160525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea28719-073d-4878-95e2-e0d6f8f8aa9f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:27.192865 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935b85ca-80cf-4dd8-85b2-52fa1ce9017b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:27.193732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b85ca-80cf-4dd8-85b2-52fa1ce9017b None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:27.194007 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b85ca-80cf-4dd8-85b2-52fa1ce9017b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:27.194276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b85ca-80cf-4dd8-85b2-52fa1ce9017b None admin] PATH_INFO: `/v3/projects/eb60b2edf7e748048eada504bb0901f5/users/90e704c3aad5448faa134f658fcaa932/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:27.196169 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935b85ca-80cf-4dd8-85b2-52fa1ce9017b None admin] RBAC: Authorizing `identity:create_grant(project_id=eb60b2edf7e748048eada504bb0901f5, user_id=90e704c3aad5448faa134f658fcaa932, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:27.197610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935b85ca-80cf-4dd8-85b2-52fa1ce9017b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:27.264259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 591/1192] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jun 23 19:50:27 2022] PUT /identity/v3/projects/eb60b2edf7e748048eada504bb0901f5/users/90e704c3aad5448faa134f658fcaa932/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:50:29.235447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b31dfc-45f7-4048-8437-313ea1c88ae2 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:29.236568 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b31dfc-45f7-4048-8437-313ea1c88ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:29.236568 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b31dfc-45f7-4048-8437-313ea1c88ae2 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:29.237918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 602/1193] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:50:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:50:29.243098 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8633e-ffa7-428d-9be0-8de75c55c6f0 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:29.243492 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8633e-ffa7-428d-9be0-8de75c55c6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:29.243655 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8633e-ffa7-428d-9be0-8de75c55c6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:29.259863 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7fb8633e-ffa7-428d-9be0-8de75c55c6f0 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:50:29.261092 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7fb8633e-ffa7-428d-9be0-8de75c55c6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:50:29.325297 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 592/1194] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:50:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:50:29.348639 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b31dfc-45f7-4048-8437-313ea1c88ae2 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:29.446062 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b31dfc-45f7-4048-8437-313ea1c88ae2 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:29.477616 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b5017-4204-485e-a7be-8c36b29a1d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:29.478380 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b5017-4204-485e-a7be-8c36b29a1d82 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:29.478657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b5017-4204-485e-a7be-8c36b29a1d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:29.478923 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b5017-4204-485e-a7be-8c36b29a1d82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:29.489614 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b5017-4204-485e-a7be-8c36b29a1d82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:50:29.497014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-cc0b5017-4204-485e-a7be-8c36b29a1d82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:50:29.497014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:50:29.497014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:50:29.497014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:50:29.497014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:50:29.497014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:50:29.497014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:50:29.497014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:50:29.497014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:50:29.497014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:50:29.497014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:50:29.497014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:50:29.497014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:50:29.497014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:50:29.497014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:50:29.497014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:50:29.498101 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:50:29.498101 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:50:29.498101 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:50:29.498101 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:50:29.498101 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:50:29.498101 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:50:29.498101 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:50:29.498101 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:50:29.498101 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:50:29.498101 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:50:29.498101 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:50:29.498101 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:50:29.498101 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:50:29.498101 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:50:29.498101 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:50:29.498101 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:50:29.499152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:50:29.499152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:50:29.499152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:50:29.499152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:50:29.499152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:50:29.499152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:50:29.499152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:50:29.499152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:50:29.499152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:50:29.499152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:50:29.499152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:50:29.499152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:50:29.499152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 603/1195] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:50:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:50:29.504101 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8633e-ffa7-428d-9be0-8de75c55c6f0 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:29.506273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:29.600308 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8633e-ffa7-428d-9be0-8de75c55c6f0 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:29.632154 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf72e3-84d0-4a2c-811d-05fa56e791b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:29.632908 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf72e3-84d0-4a2c-811d-05fa56e791b1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:29.633193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf72e3-84d0-4a2c-811d-05fa56e791b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:29.633467 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf72e3-84d0-4a2c-811d-05fa56e791b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:29.634202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf72e3-84d0-4a2c-811d-05fa56e791b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:50:29.634506 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf72e3-84d0-4a2c-811d-05fa56e791b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:29.635867 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf72e3-84d0-4a2c-811d-05fa56e791b1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:29.644413 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 593/1196] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:50:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:50:29.649881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b5017-4204-485e-a7be-8c36b29a1d82 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:29.668162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:29.745588 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b5017-4204-485e-a7be-8c36b29a1d82 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:29.778569 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb8e38-b34c-4178-b6bc-677782225922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:29.779417 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb8e38-b34c-4178-b6bc-677782225922 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:29.779697 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb8e38-b34c-4178-b6bc-677782225922 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:29.779969 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb8e38-b34c-4178-b6bc-677782225922 None admin] PATH_INFO: `/v3/users/90e704c3aad5448faa134f658fcaa932` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:29.781158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb8e38-b34c-4178-b6bc-677782225922 None admin] RBAC: Authorizing `identity:get_user(user_id=90e704c3aad5448faa134f658fcaa932)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:29.783210 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb8e38-b34c-4178-b6bc-677782225922 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:29.784826 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 604/1197] 10.222.0.44 () {56 vars in 1177 bytes} [Thu Jun 23 19:50:29 2022] GET /identity/v3/users/90e704c3aad5448faa134f658fcaa932 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:50:29.790345 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf72e3-84d0-4a2c-811d-05fa56e791b1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:29.850294 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:29.885746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf72e3-84d0-4a2c-811d-05fa56e791b1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:29.918437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa077e5-6c16-4439-81a5-ef6e3dbb57b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:29.919065 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa077e5-6c16-4439-81a5-ef6e3dbb57b4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:29.919342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa077e5-6c16-4439-81a5-ef6e3dbb57b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:29.919609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa077e5-6c16-4439-81a5-ef6e3dbb57b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:29.982780 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa077e5-6c16-4439-81a5-ef6e3dbb57b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:29.984219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa077e5-6c16-4439-81a5-ef6e3dbb57b4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:30.041395 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:30.078377 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 594/1198] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:50:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 19:50:30.084700 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb8e38-b34c-4178-b6bc-677782225922 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:30.164140 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:30.178783 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb8e38-b34c-4178-b6bc-677782225922 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:30.212082 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2891eff2-9adf-4b6d-92ac-af27b9664c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:30.212938 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891eff2-9adf-4b6d-92ac-af27b9664c58 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:30.213216 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891eff2-9adf-4b6d-92ac-af27b9664c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:30.213481 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2891eff2-9adf-4b6d-92ac-af27b9664c58 None admin] PATH_INFO: `/v3/projects/eb60b2edf7e748048eada504bb0901f5` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:30.214627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2891eff2-9adf-4b6d-92ac-af27b9664c58 None admin] RBAC: Authorizing `identity:get_project(project_id=eb60b2edf7e748048eada504bb0901f5)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:30.216054 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2891eff2-9adf-4b6d-92ac-af27b9664c58 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:30.217711 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 605/1199] 10.222.0.44 () {56 vars in 1186 bytes} [Thu Jun 23 19:50:30 2022] GET /identity/v3/projects/eb60b2edf7e748048eada504bb0901f5 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:50:30.223610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa077e5-6c16-4439-81a5-ef6e3dbb57b4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:50:30.323625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa077e5-6c16-4439-81a5-ef6e3dbb57b4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:50:30.356706 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f0411-436a-44f6-bdee-7b22063b0f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:50:30.357345 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f0411-436a-44f6-bdee-7b22063b0f46 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:50:30.357628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f0411-436a-44f6-bdee-7b22063b0f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:50:30.357903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f0411-436a-44f6-bdee-7b22063b0f46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:50:30.358722 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f0411-436a-44f6-bdee-7b22063b0f46 None admin] RBAC: Adding query filter params (role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb, scope.project.id=eb60b2edf7e748048eada504bb0901f5, user.id=90e704c3aad5448faa134f658fcaa932) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:50:30.359034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f0411-436a-44f6-bdee-7b22063b0f46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:50:30.360974 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f0411-436a-44f6-bdee-7b22063b0f46 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:50:30.366703 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 595/1200] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:50:30 2022] GET /identity/v3/role_assignments?user.id=90e704c3aad5448faa134f658fcaa932&scope.project.id=eb60b2edf7e748048eada504bb0901f5&role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:50:40.208116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=24 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:50:40.363989 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=86 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:42.411486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba7458-d941-4d54-afb9-1fce832513fb None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:42.412109 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba7458-d941-4d54-afb9-1fce832513fb None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:42.412109 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba7458-d941-4d54-afb9-1fce832513fb None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:42.413461 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 606/1201] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:51:42.419219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fab8d-82c4-4c8e-98c5-625f7eda6224 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:42.419712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fab8d-82c4-4c8e-98c5-625f7eda6224 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:42.421598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06fab8d-82c4-4c8e-98c5-625f7eda6224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:42.439420 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a06fab8d-82c4-4c8e-98c5-625f7eda6224 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:51:42.440662 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a06fab8d-82c4-4c8e-98c5-625f7eda6224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:51:42.509099 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 596/1202] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:51:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:51:42.533943 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba7458-d941-4d54-afb9-1fce832513fb None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:42.642243 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba7458-d941-4d54-afb9-1fce832513fb None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:42.678799 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e144b26-519f-483e-98ef-9a5c8a3a5e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:42.679633 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e144b26-519f-483e-98ef-9a5c8a3a5e6d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:42.679915 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e144b26-519f-483e-98ef-9a5c8a3a5e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:42.680188 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e144b26-519f-483e-98ef-9a5c8a3a5e6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:42.690682 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e144b26-519f-483e-98ef-9a5c8a3a5e6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:42.694988 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e144b26-519f-483e-98ef-9a5c8a3a5e6d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:42.710637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-8e144b26-519f-483e-98ef-9a5c8a3a5e6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:51:42.710637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:51:42.710637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:51:42.710637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:51:42.710637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:51:42.710637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:51:42.710637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:51:42.710637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:51:42.710637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:51:42.710637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:51:42.710637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:51:42.710637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:51:42.710637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:51:42.710637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:51:42.710637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:51:42.711627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:51:42.711627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:51:42.711627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:51:42.711627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:51:42.711627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:51:42.711627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:51:42.711627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:51:42.711627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:51:42.711627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:51:42.711627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:51:42.711627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:51:42.711627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:51:42.711627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:51:42.711627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:51:42.711627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:51:42.711627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:51:42.711627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:51:42.712671 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:51:42.712671 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:51:42.712671 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:51:42.712671 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:51:42.716454 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 607/1203] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:51:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:51:42.718609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06fab8d-82c4-4c8e-98c5-625f7eda6224 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:42.818276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06fab8d-82c4-4c8e-98c5-625f7eda6224 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:42.848368 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:42.854034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16c92a-b641-4610-a38b-579554616e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:42.854685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16c92a-b641-4610-a38b-579554616e58 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:42.854963 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16c92a-b641-4610-a38b-579554616e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:42.855232 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16c92a-b641-4610-a38b-579554616e58 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:42.855940 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a16c92a-b641-4610-a38b-579554616e58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:51:42.856259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a16c92a-b641-4610-a38b-579554616e58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:42.857703 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a16c92a-b641-4610-a38b-579554616e58 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:42.868722 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 597/1204] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:51:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:51:42.874520 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e144b26-519f-483e-98ef-9a5c8a3a5e6d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:42.926604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:42.971269 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e144b26-519f-483e-98ef-9a5c8a3a5e6d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:43.006649 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565e020-d956-43d1-bc45-9d1e4f5db736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:43.007316 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565e020-d956-43d1-bc45-9d1e4f5db736 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:43.007600 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565e020-d956-43d1-bc45-9d1e4f5db736 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:43.007877 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565e020-d956-43d1-bc45-9d1e4f5db736 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:43.008624 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6565e020-d956-43d1-bc45-9d1e4f5db736 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:43.010067 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6565e020-d956-43d1-bc45-9d1e4f5db736 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:43.120098 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 608/1205] 10.222.0.44 () {60 vars in 1132 bytes} [Thu Jun 23 19:51:42 2022] POST /identity/v3/users => generated 251 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 19:51:45.164872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b808d-0e8a-4dff-a232-c2ccd72a3ba1 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:45.165433 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b808d-0e8a-4dff-a232-c2ccd72a3ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:45.165433 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b808d-0e8a-4dff-a232-c2ccd72a3ba1 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:45.166861 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 598/1206] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:51:45.172599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6999f1-514b-4f6c-a4dc-78e0abd7f135 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:45.173132 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6999f1-514b-4f6c-a4dc-78e0abd7f135 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:45.173612 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6999f1-514b-4f6c-a4dc-78e0abd7f135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:45.193218 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-be6999f1-514b-4f6c-a4dc-78e0abd7f135 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:51:45.194429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-be6999f1-514b-4f6c-a4dc-78e0abd7f135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:51:45.268628 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 609/1207] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:51:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:51:45.294085 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b808d-0e8a-4dff-a232-c2ccd72a3ba1 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:45.394657 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b808d-0e8a-4dff-a232-c2ccd72a3ba1 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:45.428343 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67f204-7139-4a8d-95eb-61ef9b38f3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:45.429073 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67f204-7139-4a8d-95eb-61ef9b38f3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:45.429352 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67f204-7139-4a8d-95eb-61ef9b38f3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:45.429623 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67f204-7139-4a8d-95eb-61ef9b38f3c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:45.439725 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f67f204-7139-4a8d-95eb-61ef9b38f3c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:51:45.447331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-1f67f204-7139-4a8d-95eb-61ef9b38f3c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:51:45.447331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:51:45.447331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:51:45.447331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:51:45.447331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:51:45.447331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:51:45.447331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:51:45.447331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:51:45.447331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:51:45.447331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:51:45.447331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:51:45.447331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:51:45.447331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:51:45.447331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:51:45.447331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:51:45.447331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:51:45.450184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:51:45.450184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:51:45.450184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:51:45.450184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:51:45.450184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:51:45.450184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:51:45.450184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:51:45.450184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:51:45.450184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:51:45.450184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:51:45.450184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:51:45.450184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:51:45.450184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:51:45.450184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:51:45.450184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:51:45.450184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:51:45.451219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:51:45.451219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:51:45.451219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:51:45.451219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:51:45.451219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:51:45.451219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:51:45.451219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:51:45.451219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:51:45.451219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:51:45.451219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:51:45.451219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:51:45.451219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:51:45.451219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 599/1208] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jun 23 19:51:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:51:45.454477 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6999f1-514b-4f6c-a4dc-78e0abd7f135 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:45.493686 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:45.550696 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6999f1-514b-4f6c-a4dc-78e0abd7f135 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:45.583543 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd53be-a340-49de-89c3-7a59dc26735f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:45.584200 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd53be-a340-49de-89c3-7a59dc26735f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:45.584512 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd53be-a340-49de-89c3-7a59dc26735f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:45.584789 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd53be-a340-49de-89c3-7a59dc26735f None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:45.585532 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dd53be-a340-49de-89c3-7a59dc26735f None admin] RBAC: Adding query filter params (name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:51:45.585845 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dd53be-a340-49de-89c3-7a59dc26735f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:45.587225 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dd53be-a340-49de-89c3-7a59dc26735f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:45.595329 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 610/1209] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jun 23 19:51:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:51:45.601007 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67f204-7139-4a8d-95eb-61ef9b38f3c9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:45.651805 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:45.702860 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67f204-7139-4a8d-95eb-61ef9b38f3c9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:45.735847 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92073ca5-dbaa-4452-aefb-cc91026bc7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:45.736658 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92073ca5-dbaa-4452-aefb-cc91026bc7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:45.736941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92073ca5-dbaa-4452-aefb-cc91026bc7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:45.737219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92073ca5-dbaa-4452-aefb-cc91026bc7c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:45.747431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92073ca5-dbaa-4452-aefb-cc91026bc7c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:45.748925 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92073ca5-dbaa-4452-aefb-cc91026bc7c5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:45.765840 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-92073ca5-dbaa-4452-aefb-cc91026bc7c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:51:45.765840 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:51:45.765840 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:51:45.765840 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:51:45.765840 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:51:45.765840 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:51:45.765840 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:51:45.765840 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:51:45.765840 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:51:45.765840 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:51:45.765840 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:51:45.765840 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:51:45.765840 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:51:45.765840 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:51:45.765840 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:51:45.767639 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:51:45.767639 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:51:45.767639 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:51:45.767639 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:51:45.767639 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:51:45.767639 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:51:45.767639 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:51:45.767639 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:51:45.767639 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:51:45.767639 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:51:45.767639 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:51:45.767639 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:51:45.767639 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:51:45.767639 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:51:45.767639 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:51:45.767639 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:51:45.767639 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:51:45.768672 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:51:45.768672 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:51:45.768672 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:51:45.768672 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:51:45.768672 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 600/1210] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:51:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:51:45.772777 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd53be-a340-49de-89c3-7a59dc26735f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:45.868144 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:45.871042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dd53be-a340-49de-89c3-7a59dc26735f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:45.902978 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afbff48-0fa6-40f9-aef8-09610381816c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:45.903626 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afbff48-0fa6-40f9-aef8-09610381816c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:45.903907 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afbff48-0fa6-40f9-aef8-09610381816c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:45.904207 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afbff48-0fa6-40f9-aef8-09610381816c None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:45.904949 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afbff48-0fa6-40f9-aef8-09610381816c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:51:45.905253 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afbff48-0fa6-40f9-aef8-09610381816c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:45.906619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afbff48-0fa6-40f9-aef8-09610381816c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:45.917815 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 611/1211] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:51:45.923944 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92073ca5-dbaa-4452-aefb-cc91026bc7c5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:46.021764 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92073ca5-dbaa-4452-aefb-cc91026bc7c5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:46.023218 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:46.054826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a456e3-2b17-40b2-9765-217454cf63ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:46.055616 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a456e3-2b17-40b2-9765-217454cf63ad None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:46.055901 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a456e3-2b17-40b2-9765-217454cf63ad None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:46.056173 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a456e3-2b17-40b2-9765-217454cf63ad None admin] PATH_INFO: `/v3/users/placement` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:46.063911 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a456e3-2b17-40b2-9765-217454cf63ad None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:46.065390 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a456e3-2b17-40b2-9765-217454cf63ad None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:46.079628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-34a456e3-2b17-40b2-9765-217454cf63ad None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 19:51:46.079628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:51:46.079628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:51:46.079628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:51:46.079628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:51:46.079628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:51:46.079628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:51:46.079628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:51:46.079628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:51:46.079628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:51:46.079628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:51:46.079628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:51:46.079628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:51:46.079628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:51:46.079628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:51:46.079628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:51:46.080803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:51:46.080803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:51:46.080803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:51:46.080803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:51:46.080803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:51:46.080803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:51:46.080803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:51:46.080803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:51:46.080803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:51:46.080803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:51:46.080803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:51:46.080803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:51:46.080803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:51:46.080803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:51:46.080803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:51:46.080803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:51:46.080803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:51:46.080803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:51:46.081962 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:51:46.081962 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:51:46.081962 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:51:46.081962 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:51:46.081962 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:51:46.081962 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:51:46.081962 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:51:46.081962 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:51:46.081962 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 19:51:46.081962 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:51:46.081962 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 601/1212] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:51:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:51:46.086831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afbff48-0fa6-40f9-aef8-09610381816c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:46.183513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afbff48-0fa6-40f9-aef8-09610381816c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:46.218698 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d577dd9-06fd-415f-a935-55a544872743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:46.219379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d577dd9-06fd-415f-a935-55a544872743 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:46.219664 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d577dd9-06fd-415f-a935-55a544872743 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:46.219939 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d577dd9-06fd-415f-a935-55a544872743 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:46.220811 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d577dd9-06fd-415f-a935-55a544872743 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:51:46.221121 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d577dd9-06fd-415f-a935-55a544872743 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:46.222569 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d577dd9-06fd-415f-a935-55a544872743 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:46.240581 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 612/1213] 10.222.0.44 () {56 vars in 1143 bytes} [Thu Jun 23 19:51:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:51:46.246192 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a456e3-2b17-40b2-9765-217454cf63ad None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:46.343850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a456e3-2b17-40b2-9765-217454cf63ad None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:46.376859 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131d08c-406e-4880-ab8d-9aa9718b3e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:46.377494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131d08c-406e-4880-ab8d-9aa9718b3e6b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:46.377776 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131d08c-406e-4880-ab8d-9aa9718b3e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:46.378049 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131d08c-406e-4880-ab8d-9aa9718b3e6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:46.407263 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:46.443949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c131d08c-406e-4880-ab8d-9aa9718b3e6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:46.445423 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c131d08c-406e-4880-ab8d-9aa9718b3e6b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:46.541130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 602/1214] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:51:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 19:51:46.547575 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d577dd9-06fd-415f-a935-55a544872743 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:46.566514 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:46.642267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d577dd9-06fd-415f-a935-55a544872743 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:46.675919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af607e-a64c-4736-ad86-61e39128817a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:46.676764 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af607e-a64c-4736-ad86-61e39128817a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:46.677043 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af607e-a64c-4736-ad86-61e39128817a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:46.677315 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af607e-a64c-4736-ad86-61e39128817a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:46.687811 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37af607e-a64c-4736-ad86-61e39128817a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:46.689302 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37af607e-a64c-4736-ad86-61e39128817a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:46.706120 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-37af607e-a64c-4736-ad86-61e39128817a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:51:46.706120 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:51:46.706120 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:51:46.706120 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:51:46.706120 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:51:46.706120 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:51:46.706120 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:51:46.706120 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:51:46.706120 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:51:46.706120 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:51:46.706120 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:51:46.706120 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:51:46.706120 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:51:46.706120 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:51:46.706120 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:51:46.707166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:51:46.707166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:51:46.707166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:51:46.707166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:51:46.707166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:51:46.707166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:51:46.707166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:51:46.707166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:51:46.707166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:51:46.707166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:51:46.707166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:51:46.707166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:51:46.707166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:51:46.707166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:51:46.707166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:51:46.707166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:51:46.707166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:51:46.708119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:51:46.708119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:51:46.708119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:51:46.708119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:51:46.708119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 613/1215] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:51:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:51:46.713036 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131d08c-406e-4880-ab8d-9aa9718b3e6b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:46.771917 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:46.809280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131d08c-406e-4880-ab8d-9aa9718b3e6b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:46.841439 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe98a4-b875-40d9-92bb-139ace601ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:46.842607 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe98a4-b875-40d9-92bb-139ace601ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:46.842883 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe98a4-b875-40d9-92bb-139ace601ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:46.843150 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe98a4-b875-40d9-92bb-139ace601ec4 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:46.843851 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abe98a4-b875-40d9-92bb-139ace601ec4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:51:46.844207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abe98a4-b875-40d9-92bb-139ace601ec4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:46.845597 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abe98a4-b875-40d9-92bb-139ace601ec4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:46.856779 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 603/1216] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:51:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:51:46.862888 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af607e-a64c-4736-ad86-61e39128817a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:46.930720 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:46.959382 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af607e-a64c-4736-ad86-61e39128817a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:46.992229 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336db255-0978-4271-8774-844a346aa649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:46.993034 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336db255-0978-4271-8774-844a346aa649 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:46.993316 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336db255-0978-4271-8774-844a346aa649 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:46.993590 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336db255-0978-4271-8774-844a346aa649 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:47.000084 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336db255-0978-4271-8774-844a346aa649 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:47.001574 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336db255-0978-4271-8774-844a346aa649 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:47.015421 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-336db255-0978-4271-8774-844a346aa649 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:51:47.015421 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:51:47.015421 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:51:47.015421 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:51:47.015421 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:51:47.015421 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:51:47.015421 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:51:47.015421 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:51:47.015421 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:51:47.015421 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:51:47.015421 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:51:47.015421 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:51:47.015421 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:51:47.015421 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:51:47.015421 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:51:47.016511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:51:47.016511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:51:47.016511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:51:47.016511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:51:47.016511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:51:47.016511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:51:47.016511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:51:47.016511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:51:47.016511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:51:47.016511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:51:47.016511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:51:47.016511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:51:47.016511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:51:47.016511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:51:47.016511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:51:47.016511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:51:47.016511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:51:47.017432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:51:47.017432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:51:47.017432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:51:47.017432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:51:47.017432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:51:47.017432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:51:47.017432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:51:47.017432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:51:47.017432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 614/1217] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:51:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:51:47.022943 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe98a4-b875-40d9-92bb-139ace601ec4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:47.119945 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe98a4-b875-40d9-92bb-139ace601ec4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:47.141520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:47.152654 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61702e54-b32f-4278-afe8-6c38b7ba4e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:47.153300 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61702e54-b32f-4278-afe8-6c38b7ba4e61 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:47.153583 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61702e54-b32f-4278-afe8-6c38b7ba4e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:47.153857 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61702e54-b32f-4278-afe8-6c38b7ba4e61 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:47.155127 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61702e54-b32f-4278-afe8-6c38b7ba4e61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:51:47.155435 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61702e54-b32f-4278-afe8-6c38b7ba4e61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:47.156887 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61702e54-b32f-4278-afe8-6c38b7ba4e61 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:47.168163 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 604/1218] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:51:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:51:47.174537 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336db255-0978-4271-8774-844a346aa649 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:47.276796 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336db255-0978-4271-8774-844a346aa649 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:47.304456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:47.310255 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746300db-4b06-4bfa-9e9a-aaa3f9c64239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:47.310906 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746300db-4b06-4bfa-9e9a-aaa3f9c64239 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:47.311189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746300db-4b06-4bfa-9e9a-aaa3f9c64239 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:47.311464 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746300db-4b06-4bfa-9e9a-aaa3f9c64239 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:47.312600 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746300db-4b06-4bfa-9e9a-aaa3f9c64239 None admin] RBAC: Adding query filter params (role.id=5f27c72acb19431fa3223c962bc42938, scope.project.id=155180ba63c446ca889a6eb3f94cc31a, user.id=9715d5deeccd419b97cb0bd39e17982f) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:51:47.312915 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746300db-4b06-4bfa-9e9a-aaa3f9c64239 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:47.314437 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746300db-4b06-4bfa-9e9a-aaa3f9c64239 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:47.321674 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 615/1219] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:51:47 2022] GET /identity/v3/role_assignments?user.id=9715d5deeccd419b97cb0bd39e17982f&scope.project.id=155180ba63c446ca889a6eb3f94cc31a&role.id=5f27c72acb19431fa3223c962bc42938 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:51:49.297899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566333ff-fd29-427d-b118-92c65ff91ef9 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:49.298434 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566333ff-fd29-427d-b118-92c65ff91ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:49.298543 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566333ff-fd29-427d-b118-92c65ff91ef9 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:49.299850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 605/1220] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:51:49.305030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6fab7-ae50-400b-b72f-92b91cd9f737 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:49.305408 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6fab7-ae50-400b-b72f-92b91cd9f737 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:49.305582 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6fab7-ae50-400b-b72f-92b91cd9f737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:49.323522 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-07a6fab7-ae50-400b-b72f-92b91cd9f737 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:51:49.324777 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-07a6fab7-ae50-400b-b72f-92b91cd9f737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:51:49.391522 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 616/1221] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:51:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:51:49.414020 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566333ff-fd29-427d-b118-92c65ff91ef9 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:49.513947 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566333ff-fd29-427d-b118-92c65ff91ef9 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:49.547144 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86665796-c95b-43cc-a8d2-ac1b8171404c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:49.547926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86665796-c95b-43cc-a8d2-ac1b8171404c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:49.548211 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86665796-c95b-43cc-a8d2-ac1b8171404c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:49.548541 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86665796-c95b-43cc-a8d2-ac1b8171404c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:49.558405 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86665796-c95b-43cc-a8d2-ac1b8171404c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:51:49.566856 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-86665796-c95b-43cc-a8d2-ac1b8171404c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:51:49.566856 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:51:49.566856 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:51:49.566856 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:51:49.566856 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:51:49.566856 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:51:49.566856 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:51:49.566856 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:51:49.566856 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:51:49.566856 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:51:49.566856 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:51:49.566856 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:51:49.566856 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:51:49.566856 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:51:49.566856 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:51:49.566856 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:51:49.571330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:51:49.571330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:51:49.571330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:51:49.571330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:51:49.571330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:51:49.571330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:51:49.571330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:51:49.571330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:51:49.571330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:51:49.571330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:51:49.571330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:51:49.571330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:51:49.571330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:51:49.571330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:51:49.571330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:51:49.571330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:51:49.572426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:51:49.572426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:51:49.572426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:51:49.572426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:51:49.572426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:51:49.572426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:51:49.572426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:51:49.572426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:51:49.572426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:51:49.572426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:51:49.572426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:51:49.572426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:51:49.572426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 606/1222] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jun 23 19:51:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:51:49.574326 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6fab7-ae50-400b-b72f-92b91cd9f737 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:49.675510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a6fab7-ae50-400b-b72f-92b91cd9f737 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:49.704160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:49.717189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3436b4-4f1a-4d59-a760-6fef99b3b35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:49.717904 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3436b4-4f1a-4d59-a760-6fef99b3b35a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:49.718219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3436b4-4f1a-4d59-a760-6fef99b3b35a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:49.718564 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3436b4-4f1a-4d59-a760-6fef99b3b35a None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:49.719498 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3436b4-4f1a-4d59-a760-6fef99b3b35a None admin] RBAC: Adding query filter params (name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:51:49.719871 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3436b4-4f1a-4d59-a760-6fef99b3b35a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:49.721437 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3436b4-4f1a-4d59-a760-6fef99b3b35a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:49.729906 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 617/1223] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jun 23 19:51:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:51:49.736256 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86665796-c95b-43cc-a8d2-ac1b8171404c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:49.779140 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:49.835984 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86665796-c95b-43cc-a8d2-ac1b8171404c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:49.868457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795275e0-5b22-489b-adb8-2094e1fb6245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:49.869252 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795275e0-5b22-489b-adb8-2094e1fb6245 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:49.869529 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795275e0-5b22-489b-adb8-2094e1fb6245 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:49.869808 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795275e0-5b22-489b-adb8-2094e1fb6245 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:49.879718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795275e0-5b22-489b-adb8-2094e1fb6245 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:49.881173 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795275e0-5b22-489b-adb8-2094e1fb6245 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:49.897281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-795275e0-5b22-489b-adb8-2094e1fb6245 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:51:49.897281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:51:49.897281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:51:49.897281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:51:49.897281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:51:49.897281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:51:49.897281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:51:49.897281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:51:49.897281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:51:49.897281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:51:49.897281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:51:49.897281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:51:49.897281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:51:49.897281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:51:49.897281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:51:49.900202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:51:49.900202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:51:49.900202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:51:49.900202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:51:49.900202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:51:49.900202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:51:49.900202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:51:49.900202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:51:49.900202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:51:49.900202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:51:49.900202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:51:49.900202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:51:49.900202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:51:49.900202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:51:49.900202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:51:49.900202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:51:49.900202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:51:49.901426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:51:49.901426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:51:49.901426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:51:49.901426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:51:49.901426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 607/1224] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:51:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:51:49.904977 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3436b4-4f1a-4d59-a760-6fef99b3b35a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:50.004723 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3436b4-4f1a-4d59-a760-6fef99b3b35a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:50.044783 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ff5f2-a299-4441-b95f-e315b1ac596b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:50.046635 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ff5f2-a299-4441-b95f-e315b1ac596b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:50.046920 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ff5f2-a299-4441-b95f-e315b1ac596b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:50.047193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ff5f2-a299-4441-b95f-e315b1ac596b None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:50.047909 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ff5f2-a299-4441-b95f-e315b1ac596b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:51:50.048214 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ff5f2-a299-4441-b95f-e315b1ac596b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:50.049695 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ff5f2-a299-4441-b95f-e315b1ac596b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:50.061139 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 618/1225] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:51:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:51:50.067091 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795275e0-5b22-489b-adb8-2094e1fb6245 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:50.157955 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:50.164725 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795275e0-5b22-489b-adb8-2094e1fb6245 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:50.198374 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ded85c-ee91-4693-8f73-0a6a7a260f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:50.199207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ded85c-ee91-4693-8f73-0a6a7a260f2a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:50.199485 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ded85c-ee91-4693-8f73-0a6a7a260f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:50.199754 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ded85c-ee91-4693-8f73-0a6a7a260f2a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:50.206910 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ded85c-ee91-4693-8f73-0a6a7a260f2a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:50.208426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ded85c-ee91-4693-8f73-0a6a7a260f2a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:50.223186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-f1ded85c-ee91-4693-8f73-0a6a7a260f2a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 19:51:50.223186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:51:50.223186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:51:50.223186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:51:50.223186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:51:50.223186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:51:50.223186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:51:50.223186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:51:50.223186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:51:50.223186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:51:50.223186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:51:50.223186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:51:50.223186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:51:50.223186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:51:50.223186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:51:50.223186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:51:50.224258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:51:50.224258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:51:50.224258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:51:50.224258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:51:50.224258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:51:50.224258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:51:50.224258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:51:50.224258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:51:50.224258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:51:50.224258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:51:50.224258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:51:50.224258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:51:50.224258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:51:50.224258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:51:50.224258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:51:50.224258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:51:50.224258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:51:50.224258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:51:50.230365 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:51:50.230365 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:51:50.230365 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:51:50.230365 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:51:50.230365 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:51:50.230365 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:51:50.230365 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:51:50.230365 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:51:50.230365 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 19:51:50.230365 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:51:50.230365 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 608/1226] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:51:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 19:51:50.230954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ff5f2-a299-4441-b95f-e315b1ac596b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:50.278101 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:50.330582 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ff5f2-a299-4441-b95f-e315b1ac596b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:50.364225 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210be579-4bb8-460e-8341-68fe4a9a76fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:50.364973 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210be579-4bb8-460e-8341-68fe4a9a76fc None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:50.365451 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210be579-4bb8-460e-8341-68fe4a9a76fc None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:50.365950 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210be579-4bb8-460e-8341-68fe4a9a76fc None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:50.366878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210be579-4bb8-460e-8341-68fe4a9a76fc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:51:50.367719 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210be579-4bb8-460e-8341-68fe4a9a76fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:50.369342 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210be579-4bb8-460e-8341-68fe4a9a76fc None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:50.385861 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 619/1227] 10.222.0.44 () {56 vars in 1143 bytes} [Thu Jun 23 19:51:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:51:50.391987 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ded85c-ee91-4693-8f73-0a6a7a260f2a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:50.489041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ded85c-ee91-4693-8f73-0a6a7a260f2a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:50.523561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca6fcf-d1cd-45e4-8fde-db912e911689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:50.524366 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca6fcf-d1cd-45e4-8fde-db912e911689 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:50.524652 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca6fcf-d1cd-45e4-8fde-db912e911689 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:50.524927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca6fcf-d1cd-45e4-8fde-db912e911689 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:50.534691 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca6fcf-d1cd-45e4-8fde-db912e911689 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:50.536163 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca6fcf-d1cd-45e4-8fde-db912e911689 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:50.553545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-e5ca6fcf-d1cd-45e4-8fde-db912e911689 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:51:50.553545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:51:50.553545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:51:50.553545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:51:50.553545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:51:50.553545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:51:50.553545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:51:50.553545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:51:50.553545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:51:50.553545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:51:50.553545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:51:50.553545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:51:50.553545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:51:50.553545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:51:50.553545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:51:50.556261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:51:50.556261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:51:50.556261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:51:50.556261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:51:50.556261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:51:50.556261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:51:50.556261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:51:50.556261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:51:50.556261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:51:50.556261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:51:50.556261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:51:50.556261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:51:50.556261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:51:50.556261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:51:50.556261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:51:50.556261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:51:50.556261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:51:50.557231 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:51:50.557231 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:51:50.557231 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:51:50.557231 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:51:50.557231 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 609/1228] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:51:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:51:50.561125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210be579-4bb8-460e-8341-68fe4a9a76fc None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:50.657885 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210be579-4bb8-460e-8341-68fe4a9a76fc None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:50.660045 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:50.691576 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024cfae1-2815-4627-a180-ce41297e8c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:50.692229 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024cfae1-2815-4627-a180-ce41297e8c7b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:50.692540 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024cfae1-2815-4627-a180-ce41297e8c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:50.692840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024cfae1-2815-4627-a180-ce41297e8c7b None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:50.693553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024cfae1-2815-4627-a180-ce41297e8c7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:51:50.693857 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024cfae1-2815-4627-a180-ce41297e8c7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:50.695233 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024cfae1-2815-4627-a180-ce41297e8c7b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:50.706023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 620/1229] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:51:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:51:50.711716 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca6fcf-d1cd-45e4-8fde-db912e911689 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:50.721177 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:50.808207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca6fcf-d1cd-45e4-8fde-db912e911689 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:50.839815 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c09fa-2dfa-450c-8edb-07c60bb1534e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:50.840615 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c09fa-2dfa-450c-8edb-07c60bb1534e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:50.840895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c09fa-2dfa-450c-8edb-07c60bb1534e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:50.841163 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c09fa-2dfa-450c-8edb-07c60bb1534e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:50.848090 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c09fa-2dfa-450c-8edb-07c60bb1534e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:50.849662 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c09fa-2dfa-450c-8edb-07c60bb1534e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:50.865137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-d55c09fa-2dfa-450c-8edb-07c60bb1534e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:51:50.865137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:51:50.865137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:51:50.865137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:51:50.865137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:51:50.865137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:51:50.865137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:51:50.865137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:51:50.865137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:51:50.865137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:51:50.865137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:51:50.865137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:51:50.865137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:51:50.865137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:51:50.865137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:51:50.866204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:51:50.866204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:51:50.866204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:51:50.866204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:51:50.866204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:51:50.866204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:51:50.866204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:51:50.866204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:51:50.866204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:51:50.866204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:51:50.866204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:51:50.866204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:51:50.866204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:51:50.866204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:51:50.866204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:51:50.866204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:51:50.866204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:51:50.867373 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:51:50.867373 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:51:50.867373 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:51:50.867373 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:51:50.867373 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:51:50.867373 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:51:50.867373 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:51:50.867373 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:51:50.867373 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 610/1230] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:51:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:51:50.872359 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024cfae1-2815-4627-a180-ce41297e8c7b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:50.969472 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024cfae1-2815-4627-a180-ce41297e8c7b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:51.000571 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3df92a-f345-4858-803d-c80ac873ef41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:51.001210 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3df92a-f345-4858-803d-c80ac873ef41 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:51.001487 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3df92a-f345-4858-803d-c80ac873ef41 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:51.001756 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3df92a-f345-4858-803d-c80ac873ef41 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:51.002523 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3df92a-f345-4858-803d-c80ac873ef41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:51:51.002826 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3df92a-f345-4858-803d-c80ac873ef41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:51.004258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3df92a-f345-4858-803d-c80ac873ef41 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:51.016153 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 621/1231] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:51:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:51:51.022226 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c09fa-2dfa-450c-8edb-07c60bb1534e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:51.086188 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:51.119689 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c09fa-2dfa-450c-8edb-07c60bb1534e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:51.151625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93272b33-3d45-443e-b5cf-becf207f04c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:51.152525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93272b33-3d45-443e-b5cf-becf207f04c3 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:51.152806 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93272b33-3d45-443e-b5cf-becf207f04c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:51.153079 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93272b33-3d45-443e-b5cf-becf207f04c3 None admin] PATH_INFO: `/v3/projects/155180ba63c446ca889a6eb3f94cc31a/users/9715d5deeccd419b97cb0bd39e17982f/roles/5f27c72acb19431fa3223c962bc42938` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:51.167088 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93272b33-3d45-443e-b5cf-becf207f04c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=155180ba63c446ca889a6eb3f94cc31a, user_id=9715d5deeccd419b97cb0bd39e17982f, role_id=5f27c72acb19431fa3223c962bc42938)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:51.168562 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93272b33-3d45-443e-b5cf-becf207f04c3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:51.292283 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 611/1232] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jun 23 19:51:51 2022] PUT /identity/v3/projects/155180ba63c446ca889a6eb3f94cc31a/users/9715d5deeccd419b97cb0bd39e17982f/roles/5f27c72acb19431fa3223c962bc42938 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:51:53.294617 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c7343-e4dc-41ad-8a85-11c88473ed0e None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:53.295164 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c7343-e4dc-41ad-8a85-11c88473ed0e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:53.295164 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c7343-e4dc-41ad-8a85-11c88473ed0e None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:53.296580 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 622/1233] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 19:51:53.301837 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da1217-5c94-45f6-b681-63e646c49ce5 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:53.302230 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da1217-5c94-45f6-b681-63e646c49ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:53.302391 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da1217-5c94-45f6-b681-63e646c49ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:53.319379 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-09da1217-5c94-45f6-b681-63e646c49ce5 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:51:53.320595 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-09da1217-5c94-45f6-b681-63e646c49ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:51:53.384913 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 612/1234] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:51:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:51:53.409693 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c7343-e4dc-41ad-8a85-11c88473ed0e None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:53.434086 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:53.506519 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c7343-e4dc-41ad-8a85-11c88473ed0e None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:53.539200 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f4b00-1dc9-4a84-a1fe-b3b50f975606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:53.539993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f4b00-1dc9-4a84-a1fe-b3b50f975606 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:53.540278 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f4b00-1dc9-4a84-a1fe-b3b50f975606 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:53.540577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f4b00-1dc9-4a84-a1fe-b3b50f975606 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:53.550420 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f4b00-1dc9-4a84-a1fe-b3b50f975606 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:51:53.558158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-c26f4b00-1dc9-4a84-a1fe-b3b50f975606 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:51:53.558158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:51:53.558158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:51:53.558158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:51:53.558158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:51:53.558158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:51:53.558158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:51:53.558158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:51:53.558158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:51:53.558158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:51:53.558158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:51:53.558158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:51:53.558158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:51:53.558158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:51:53.558158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:51:53.558158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:51:53.559248 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:51:53.559248 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:51:53.559248 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:51:53.559248 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:51:53.559248 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:51:53.559248 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:51:53.559248 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:51:53.559248 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:51:53.559248 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:51:53.559248 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:51:53.559248 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:51:53.559248 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:51:53.559248 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:51:53.559248 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:51:53.559248 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:51:53.559248 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:51:53.560133 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:51:53.560133 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:51:53.560133 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:51:53.560133 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:51:53.560133 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:51:53.560133 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:51:53.560133 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:51:53.560133 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:51:53.560133 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:51:53.560133 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:51:53.560133 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 19:51:53.560133 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:51:53.560133 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 623/1235] 10.222.0.44 () {56 vars in 1102 bytes} [Thu Jun 23 19:51:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:51:53.565579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da1217-5c94-45f6-b681-63e646c49ce5 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:53.611148 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:53.662500 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da1217-5c94-45f6-b681-63e646c49ce5 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:53.695512 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4067228-2922-48bf-9604-2f0163670bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:53.696174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4067228-2922-48bf-9604-2f0163670bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:53.696485 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4067228-2922-48bf-9604-2f0163670bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:53.696762 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4067228-2922-48bf-9604-2f0163670bc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:53.697506 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4067228-2922-48bf-9604-2f0163670bc7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:51:53.697814 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4067228-2922-48bf-9604-2f0163670bc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:53.699191 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4067228-2922-48bf-9604-2f0163670bc7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:53.707435 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 613/1236] 10.222.0.44 () {56 vars in 1103 bytes} [Thu Jun 23 19:51:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:51:53.713413 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f4b00-1dc9-4a84-a1fe-b3b50f975606 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:53.795594 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:53.810891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f4b00-1dc9-4a84-a1fe-b3b50f975606 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:53.843005 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20252f-ae9e-4ed4-81cd-554050ef3cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:53.843782 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20252f-ae9e-4ed4-81cd-554050ef3cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:53.844065 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20252f-ae9e-4ed4-81cd-554050ef3cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:53.844366 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20252f-ae9e-4ed4-81cd-554050ef3cd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:53.854368 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c20252f-ae9e-4ed4-81cd-554050ef3cd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:53.855824 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c20252f-ae9e-4ed4-81cd-554050ef3cd1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:53.872846 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-7c20252f-ae9e-4ed4-81cd-554050ef3cd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:51:53.872846 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:51:53.872846 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:51:53.872846 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:51:53.872846 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:51:53.872846 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:51:53.872846 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:51:53.872846 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:51:53.872846 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:51:53.872846 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:51:53.872846 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:51:53.872846 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:51:53.872846 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:51:53.872846 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:51:53.872846 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:51:53.874432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:51:53.874432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:51:53.874432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:51:53.874432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:51:53.874432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:51:53.874432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:51:53.874432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:51:53.874432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:51:53.874432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:51:53.874432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:51:53.874432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:51:53.874432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:51:53.874432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:51:53.874432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:51:53.874432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:51:53.874432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:51:53.874432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:51:53.875559 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:51:53.875559 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:51:53.875559 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:51:53.875559 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:51:53.875559 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 624/1237] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:51:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:51:53.880452 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4067228-2922-48bf-9604-2f0163670bc7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:53.981450 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4067228-2922-48bf-9604-2f0163670bc7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:53.990180 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:54.014398 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af733060-3b47-416d-9390-150978e828b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:54.015045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af733060-3b47-416d-9390-150978e828b3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:54.015328 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af733060-3b47-416d-9390-150978e828b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:54.015603 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af733060-3b47-416d-9390-150978e828b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:54.016339 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af733060-3b47-416d-9390-150978e828b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:51:54.016649 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af733060-3b47-416d-9390-150978e828b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:54.018021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af733060-3b47-416d-9390-150978e828b3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:54.030270 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 614/1238] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:51:54.035997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20252f-ae9e-4ed4-81cd-554050ef3cd1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:54.132836 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20252f-ae9e-4ed4-81cd-554050ef3cd1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:54.166944 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f2f7a-2632-422e-ad73-0b89d88cae6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:54.167730 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f2f7a-2632-422e-ad73-0b89d88cae6a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:54.168013 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f2f7a-2632-422e-ad73-0b89d88cae6a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:54.168286 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f2f7a-2632-422e-ad73-0b89d88cae6a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:54.170923 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:54.175739 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f2f7a-2632-422e-ad73-0b89d88cae6a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:54.177218 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f2f7a-2632-422e-ad73-0b89d88cae6a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:54.191274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-e89f2f7a-2632-422e-ad73-0b89d88cae6a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 19:51:54.191274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:51:54.191274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:51:54.191274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:51:54.191274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:51:54.191274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:51:54.191274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:51:54.191274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:51:54.191274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:51:54.191274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:51:54.191274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:51:54.191274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:51:54.191274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:51:54.191274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:51:54.191274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:51:54.191274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:51:54.192391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:51:54.192391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:51:54.192391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:51:54.192391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:51:54.192391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:51:54.192391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:51:54.192391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:51:54.192391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:51:54.192391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:51:54.192391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:51:54.192391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:51:54.192391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:51:54.192391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:51:54.192391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:51:54.192391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:51:54.192391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:51:54.192391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:51:54.192391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:51:54.193511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:51:54.193511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:51:54.193511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:51:54.193511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:51:54.193511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:51:54.193511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:51:54.193511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:51:54.193511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:51:54.193511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 19:51:54.193511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:51:54.193511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 625/1239] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:51:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:51:54.198179 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af733060-3b47-416d-9390-150978e828b3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:54.298046 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af733060-3b47-416d-9390-150978e828b3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:54.331655 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125e444-5721-4b6d-824a-99d20fa16dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:54.332331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125e444-5721-4b6d-824a-99d20fa16dbd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:54.332614 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125e444-5721-4b6d-824a-99d20fa16dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:54.332886 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125e444-5721-4b6d-824a-99d20fa16dbd None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:54.333861 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125e444-5721-4b6d-824a-99d20fa16dbd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:51:54.334168 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125e444-5721-4b6d-824a-99d20fa16dbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:54.335584 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125e444-5721-4b6d-824a-99d20fa16dbd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:54.353101 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 615/1240] 10.222.0.44 () {56 vars in 1143 bytes} [Thu Jun 23 19:51:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:51:54.359111 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f2f7a-2632-422e-ad73-0b89d88cae6a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:54.457731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f2f7a-2632-422e-ad73-0b89d88cae6a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:54.491005 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ac5e2-40f1-47fc-8b09-816dbb4bd03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:54.491635 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ac5e2-40f1-47fc-8b09-816dbb4bd03e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:54.491916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ac5e2-40f1-47fc-8b09-816dbb4bd03e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:54.492184 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ac5e2-40f1-47fc-8b09-816dbb4bd03e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:54.553973 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:54.560924 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ac5e2-40f1-47fc-8b09-816dbb4bd03e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:54.562658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ac5e2-40f1-47fc-8b09-816dbb4bd03e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:54.656603 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 626/1241] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:51:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 19:51:54.662455 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125e444-5721-4b6d-824a-99d20fa16dbd None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:54.691650 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:54.764249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125e444-5721-4b6d-824a-99d20fa16dbd None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:54.797047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74563a1-095f-4f5b-8595-0ce2d356f327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:54.797819 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74563a1-095f-4f5b-8595-0ce2d356f327 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:54.798172 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74563a1-095f-4f5b-8595-0ce2d356f327 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:54.798723 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74563a1-095f-4f5b-8595-0ce2d356f327 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:54.807992 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74563a1-095f-4f5b-8595-0ce2d356f327 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:54.809580 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74563a1-095f-4f5b-8595-0ce2d356f327 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:54.827833 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-c74563a1-095f-4f5b-8595-0ce2d356f327 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:51:54.827833 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:51:54.827833 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:51:54.827833 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:51:54.827833 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:51:54.827833 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:51:54.827833 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:51:54.827833 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:51:54.827833 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:51:54.827833 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:51:54.827833 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:51:54.827833 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:51:54.827833 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:51:54.827833 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:51:54.827833 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:51:54.835895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:51:54.835895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:51:54.835895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:51:54.835895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:51:54.835895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:51:54.835895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:51:54.835895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:51:54.835895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:51:54.835895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:51:54.835895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:51:54.835895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:51:54.835895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:51:54.835895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:51:54.835895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:51:54.835895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:51:54.835895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:51:54.835895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:51:54.837072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:51:54.837072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:51:54.837072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:51:54.837072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:51:54.837072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 616/1242] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:51:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:51:54.837361 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ac5e2-40f1-47fc-8b09-816dbb4bd03e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:54.926127 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:54.931813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ac5e2-40f1-47fc-8b09-816dbb4bd03e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:54.964668 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1b0a5-da98-4f80-8231-a79b84a83c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:54.965351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1b0a5-da98-4f80-8231-a79b84a83c92 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:54.965647 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1b0a5-da98-4f80-8231-a79b84a83c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:54.965904 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1b0a5-da98-4f80-8231-a79b84a83c92 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:54.966614 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f1b0a5-da98-4f80-8231-a79b84a83c92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:51:54.966939 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f1b0a5-da98-4f80-8231-a79b84a83c92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:54.968330 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f1b0a5-da98-4f80-8231-a79b84a83c92 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:54.979219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 627/1243] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:51:54.985434 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74563a1-095f-4f5b-8595-0ce2d356f327 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:55.066365 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:55.086480 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74563a1-095f-4f5b-8595-0ce2d356f327 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:55.119315 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d461b-fe63-46db-92c0-9aa7ce4fb4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:55.120091 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d461b-fe63-46db-92c0-9aa7ce4fb4ef None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:55.120395 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d461b-fe63-46db-92c0-9aa7ce4fb4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:55.120667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236d461b-fe63-46db-92c0-9aa7ce4fb4ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:55.126958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d461b-fe63-46db-92c0-9aa7ce4fb4ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:55.128430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236d461b-fe63-46db-92c0-9aa7ce4fb4ef None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:55.141916 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-236d461b-fe63-46db-92c0-9aa7ce4fb4ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:51:55.141916 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:51:55.141916 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:51:55.141916 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:51:55.141916 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:51:55.141916 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:51:55.141916 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:51:55.141916 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:51:55.141916 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:51:55.141916 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:51:55.141916 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:51:55.141916 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:51:55.141916 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:51:55.141916 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:51:55.141916 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:51:55.143870 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:51:55.143870 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:51:55.143870 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:51:55.143870 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:51:55.143870 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:51:55.143870 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:51:55.143870 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:51:55.143870 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:51:55.143870 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:51:55.143870 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:51:55.143870 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:51:55.143870 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:51:55.143870 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:51:55.143870 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:51:55.143870 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:51:55.143870 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:51:55.143870 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:51:55.144989 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:51:55.144989 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:51:55.144989 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:51:55.144989 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:51:55.144989 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:51:55.144989 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:51:55.144989 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:51:55.144989 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:51:55.144989 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 617/1244] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:51:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:51:55.148941 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1b0a5-da98-4f80-8231-a79b84a83c92 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:55.248865 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1b0a5-da98-4f80-8231-a79b84a83c92 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:55.281586 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74461a02-e49d-426d-ad83-80bec1af667a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:55.282232 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74461a02-e49d-426d-ad83-80bec1af667a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:55.282593 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74461a02-e49d-426d-ad83-80bec1af667a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:55.282869 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74461a02-e49d-426d-ad83-80bec1af667a None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:55.283643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74461a02-e49d-426d-ad83-80bec1af667a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:51:55.283948 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74461a02-e49d-426d-ad83-80bec1af667a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:55.285483 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74461a02-e49d-426d-ad83-80bec1af667a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:55.297986 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 628/1245] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:51:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:51:55.304094 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d461b-fe63-46db-92c0-9aa7ce4fb4ef None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:55.402145 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236d461b-fe63-46db-92c0-9aa7ce4fb4ef None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:55.435456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624fcb6c-0a0a-4925-bcf5-530a08781ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:55.436092 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fcb6c-0a0a-4925-bcf5-530a08781ebc None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:55.436396 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fcb6c-0a0a-4925-bcf5-530a08781ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:55.436672 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fcb6c-0a0a-4925-bcf5-530a08781ebc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:55.437834 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624fcb6c-0a0a-4925-bcf5-530a08781ebc None admin] RBAC: Adding query filter params (role.id=5f27c72acb19431fa3223c962bc42938, scope.project.id=155180ba63c446ca889a6eb3f94cc31a, user.id=9715d5deeccd419b97cb0bd39e17982f) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:51:55.438138 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624fcb6c-0a0a-4925-bcf5-530a08781ebc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:55.439623 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624fcb6c-0a0a-4925-bcf5-530a08781ebc None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:55.445454 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 618/1246] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:51:55 2022] GET /identity/v3/role_assignments?user.id=9715d5deeccd419b97cb0bd39e17982f&scope.project.id=155180ba63c446ca889a6eb3f94cc31a&role.id=5f27c72acb19431fa3223c962bc42938 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:51:57.519998 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601319b-b22b-42a2-9020-a1321ea6fc22 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:57.520523 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601319b-b22b-42a2-9020-a1321ea6fc22 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:57.520523 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4601319b-b22b-42a2-9020-a1321ea6fc22 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:57.521848 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 629/1247] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:51:57.527441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb0716-b9b7-482b-b621-7b8e271eef8b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:57.527866 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb0716-b9b7-482b-b621-7b8e271eef8b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:57.528008 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb0716-b9b7-482b-b621-7b8e271eef8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:57.533045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:57.544834 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-0ebb0716-b9b7-482b-b621-7b8e271eef8b None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:51:57.546165 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0ebb0716-b9b7-482b-b621-7b8e271eef8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:51:57.611632 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 619/1248] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:51:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:51:57.635823 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4601319b-b22b-42a2-9020-a1321ea6fc22 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:57.654040 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:57.734449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4601319b-b22b-42a2-9020-a1321ea6fc22 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:57.767292 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b032213-fdb1-49f3-86ff-6c7f3b71decf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:57.768102 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b032213-fdb1-49f3-86ff-6c7f3b71decf None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:57.768411 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b032213-fdb1-49f3-86ff-6c7f3b71decf None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:57.768709 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b032213-fdb1-49f3-86ff-6c7f3b71decf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:57.778536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b032213-fdb1-49f3-86ff-6c7f3b71decf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:51:57.785874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-6b032213-fdb1-49f3-86ff-6c7f3b71decf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:51:57.785874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:51:57.785874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:51:57.785874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:51:57.785874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:51:57.785874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:51:57.785874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:51:57.785874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:51:57.785874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:51:57.785874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:51:57.785874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:51:57.785874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:51:57.785874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:51:57.785874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:51:57.785874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:51:57.785874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:51:57.786968 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:51:57.786968 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:51:57.786968 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:51:57.786968 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:51:57.786968 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:51:57.786968 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:51:57.786968 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:51:57.786968 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:51:57.786968 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:51:57.786968 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:51:57.786968 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:51:57.786968 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:51:57.786968 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:51:57.786968 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:51:57.786968 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:51:57.786968 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:51:57.788108 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:51:57.788108 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:51:57.788108 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:51:57.788108 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:51:57.788108 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:51:57.788108 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:51:57.788108 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:51:57.788108 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:51:57.788108 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:51:57.788108 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:51:57.788108 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:51:57.788108 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:51:57.791979 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 630/1249] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:51:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:51:57.793006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb0716-b9b7-482b-b621-7b8e271eef8b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:57.888992 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb0716-b9b7-482b-b621-7b8e271eef8b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:57.917237 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:57.922374 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a5af6-c869-418c-a392-c2d4dd86daa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:57.923029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a5af6-c869-418c-a392-c2d4dd86daa5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:57.923311 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a5af6-c869-418c-a392-c2d4dd86daa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:57.923587 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a5af6-c869-418c-a392-c2d4dd86daa5 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:57.924471 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a5af6-c869-418c-a392-c2d4dd86daa5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:51:57.924783 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a5af6-c869-418c-a392-c2d4dd86daa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:57.926190 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a5af6-c869-418c-a392-c2d4dd86daa5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:57.934075 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 620/1250] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:51:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:51:57.940439 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b032213-fdb1-49f3-86ff-6c7f3b71decf None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:58.015798 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:58.036705 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b032213-fdb1-49f3-86ff-6c7f3b71decf None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:58.069095 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26524b5c-cf4e-4b05-ba8a-19429677bdc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:58.069866 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26524b5c-cf4e-4b05-ba8a-19429677bdc0 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:58.070142 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26524b5c-cf4e-4b05-ba8a-19429677bdc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:58.070409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26524b5c-cf4e-4b05-ba8a-19429677bdc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:58.080328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26524b5c-cf4e-4b05-ba8a-19429677bdc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:58.081771 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26524b5c-cf4e-4b05-ba8a-19429677bdc0 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:58.098400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-26524b5c-cf4e-4b05-ba8a-19429677bdc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:51:58.098400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:51:58.098400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:51:58.098400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:51:58.098400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:51:58.098400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:51:58.098400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:51:58.098400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:51:58.098400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:51:58.098400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:51:58.098400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:51:58.098400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:51:58.098400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:51:58.098400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:51:58.098400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:51:58.099399 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:51:58.099399 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:51:58.099399 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:51:58.099399 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:51:58.099399 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:51:58.099399 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:51:58.099399 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:51:58.099399 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:51:58.099399 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:51:58.099399 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:51:58.099399 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:51:58.099399 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:51:58.099399 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:51:58.099399 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:51:58.099399 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:51:58.099399 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:51:58.099399 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:51:58.100294 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:51:58.100294 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:51:58.100294 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:51:58.100294 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:51:58.100294 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 631/1251] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:51:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:51:58.105195 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a5af6-c869-418c-a392-c2d4dd86daa5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:58.205421 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a5af6-c869-418c-a392-c2d4dd86daa5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:58.239860 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57076fbe-7d07-4d52-95c6-dbd45ccb3bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:58.240593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57076fbe-7d07-4d52-95c6-dbd45ccb3bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:58.240879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57076fbe-7d07-4d52-95c6-dbd45ccb3bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:58.241159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57076fbe-7d07-4d52-95c6-dbd45ccb3bb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:58.241878 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57076fbe-7d07-4d52-95c6-dbd45ccb3bb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:51:58.242189 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57076fbe-7d07-4d52-95c6-dbd45ccb3bb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:58.243555 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57076fbe-7d07-4d52-95c6-dbd45ccb3bb3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:58.255272 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 621/1252] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:51:58.261045 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26524b5c-cf4e-4b05-ba8a-19429677bdc0 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:58.359896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26524b5c-cf4e-4b05-ba8a-19429677bdc0 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:58.384651 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:58.397023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c47bb-ed2a-4f23-92f5-610c0d3ee2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:58.397833 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c47bb-ed2a-4f23-92f5-610c0d3ee2f4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:58.398117 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c47bb-ed2a-4f23-92f5-610c0d3ee2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:58.398395 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c47bb-ed2a-4f23-92f5-610c0d3ee2f4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:58.406798 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c47bb-ed2a-4f23-92f5-610c0d3ee2f4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:58.408261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c47bb-ed2a-4f23-92f5-610c0d3ee2f4 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:58.422731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-970c47bb-ed2a-4f23-92f5-610c0d3ee2f4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 19:51:58.422731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:51:58.422731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:51:58.422731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:51:58.422731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:51:58.422731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:51:58.422731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:51:58.422731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:51:58.422731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:51:58.422731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:51:58.422731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:51:58.422731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:51:58.422731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:51:58.422731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:51:58.422731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:51:58.422731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:51:58.429035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:51:58.429035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:51:58.429035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:51:58.429035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:51:58.429035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:51:58.429035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:51:58.429035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:51:58.429035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:51:58.429035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:51:58.429035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:51:58.429035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:51:58.429035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:51:58.429035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:51:58.429035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:51:58.429035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:51:58.429035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:51:58.429035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:51:58.429035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:51:58.430297 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:51:58.430297 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:51:58.430297 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:51:58.430297 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:51:58.430297 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:51:58.430297 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:51:58.430297 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:51:58.430297 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:51:58.430297 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 19:51:58.430297 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:51:58.430297 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 632/1253] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:51:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:51:58.431052 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57076fbe-7d07-4d52-95c6-dbd45ccb3bb3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:58.485545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:58.527628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57076fbe-7d07-4d52-95c6-dbd45ccb3bb3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:58.560025 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e47b89-4868-428a-a1cd-b3b0bac8f0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:58.560787 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e47b89-4868-428a-a1cd-b3b0bac8f0b5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:58.561063 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e47b89-4868-428a-a1cd-b3b0bac8f0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:58.561338 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e47b89-4868-428a-a1cd-b3b0bac8f0b5 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:58.562695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e47b89-4868-428a-a1cd-b3b0bac8f0b5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:51:58.562695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e47b89-4868-428a-a1cd-b3b0bac8f0b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:58.564087 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e47b89-4868-428a-a1cd-b3b0bac8f0b5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:58.581330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 622/1254] 10.222.0.44 () {56 vars in 1143 bytes} [Thu Jun 23 19:51:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:51:58.587015 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c47bb-ed2a-4f23-92f5-610c0d3ee2f4 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:58.684807 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c47bb-ed2a-4f23-92f5-610c0d3ee2f4 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:58.716899 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e0fde-6df3-44ac-90a2-4976749b31ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:58.717531 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e0fde-6df3-44ac-90a2-4976749b31ef None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:58.717810 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e0fde-6df3-44ac-90a2-4976749b31ef None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:58.718080 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e0fde-6df3-44ac-90a2-4976749b31ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:58.770953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:58.782655 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e0fde-6df3-44ac-90a2-4976749b31ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:58.784206 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e0fde-6df3-44ac-90a2-4976749b31ef None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:58.880615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 633/1255] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:51:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 19:51:58.886701 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e47b89-4868-428a-a1cd-b3b0bac8f0b5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:58.985081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e47b89-4868-428a-a1cd-b3b0bac8f0b5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:58.998038 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:59.018345 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2af9f-4977-4a74-b4e7-f0c2840d717a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:59.019129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2af9f-4977-4a74-b4e7-f0c2840d717a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:59.020126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2af9f-4977-4a74-b4e7-f0c2840d717a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:59.020126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2af9f-4977-4a74-b4e7-f0c2840d717a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:59.029868 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2af9f-4977-4a74-b4e7-f0c2840d717a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:59.031329 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2af9f-4977-4a74-b4e7-f0c2840d717a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:59.048625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-daf2af9f-4977-4a74-b4e7-f0c2840d717a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:51:59.048625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:51:59.048625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:51:59.048625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:51:59.048625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:51:59.048625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:51:59.048625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:51:59.048625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:51:59.048625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:51:59.048625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:51:59.048625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:51:59.048625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:51:59.048625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:51:59.048625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:51:59.048625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:51:59.049769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:51:59.049769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:51:59.049769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:51:59.049769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:51:59.049769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:51:59.049769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:51:59.049769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:51:59.049769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:51:59.049769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:51:59.049769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:51:59.049769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:51:59.049769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:51:59.049769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:51:59.049769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:51:59.049769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:51:59.049769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:51:59.049769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:51:59.051218 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:51:59.051218 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:51:59.051218 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:51:59.051218 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:51:59.051218 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 623/1256] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:51:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:51:59.055989 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e0fde-6df3-44ac-90a2-4976749b31ef None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:59.140469 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:59.151853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e0fde-6df3-44ac-90a2-4976749b31ef None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:59.185255 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858cdac-2494-43f1-a708-ebe356e16eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:59.186668 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858cdac-2494-43f1-a708-ebe356e16eee None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:59.186957 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858cdac-2494-43f1-a708-ebe356e16eee None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:59.187567 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858cdac-2494-43f1-a708-ebe356e16eee None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:59.188280 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858cdac-2494-43f1-a708-ebe356e16eee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:51:59.191143 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858cdac-2494-43f1-a708-ebe356e16eee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:59.191143 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858cdac-2494-43f1-a708-ebe356e16eee None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:59.201790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 634/1257] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:51:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:51:59.207261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2af9f-4977-4a74-b4e7-f0c2840d717a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:59.319267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2af9f-4977-4a74-b4e7-f0c2840d717a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:59.353487 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82944277-f92f-4fd8-95f8-31e4ef2fa7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:59.354274 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82944277-f92f-4fd8-95f8-31e4ef2fa7e5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:59.354554 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82944277-f92f-4fd8-95f8-31e4ef2fa7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:59.354826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82944277-f92f-4fd8-95f8-31e4ef2fa7e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:59.361467 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82944277-f92f-4fd8-95f8-31e4ef2fa7e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:59.362898 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82944277-f92f-4fd8-95f8-31e4ef2fa7e5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:59.376290 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-82944277-f92f-4fd8-95f8-31e4ef2fa7e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:51:59.376290 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:51:59.376290 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:51:59.376290 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:51:59.376290 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:51:59.376290 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:51:59.376290 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:51:59.376290 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:51:59.376290 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:51:59.376290 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:51:59.376290 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:51:59.376290 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:51:59.376290 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:51:59.376290 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:51:59.376290 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:51:59.377431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:51:59.377431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:51:59.377431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:51:59.377431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:51:59.377431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:51:59.377431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:51:59.377431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:51:59.377431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:51:59.377431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:51:59.377431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:51:59.377431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:51:59.377431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:51:59.377431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:51:59.377431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:51:59.377431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:51:59.377431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:51:59.377431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:51:59.378577 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:51:59.378577 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:51:59.378577 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:51:59.378577 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:51:59.378577 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:51:59.378577 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:51:59.378577 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:51:59.378577 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:51:59.378577 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 624/1258] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:51:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:51:59.383375 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858cdac-2494-43f1-a708-ebe356e16eee None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:59.481666 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858cdac-2494-43f1-a708-ebe356e16eee None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:59.514351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c21a4-1fb1-4f1d-8df0-37b7f07166c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:59.514945 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c21a4-1fb1-4f1d-8df0-37b7f07166c5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:59.515222 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c21a4-1fb1-4f1d-8df0-37b7f07166c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:59.515492 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c21a4-1fb1-4f1d-8df0-37b7f07166c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:59.516263 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c21a4-1fb1-4f1d-8df0-37b7f07166c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:51:59.516598 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c21a4-1fb1-4f1d-8df0-37b7f07166c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:59.518027 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c21a4-1fb1-4f1d-8df0-37b7f07166c5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:59.530095 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 635/1259] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:51:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:51:59.536204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82944277-f92f-4fd8-95f8-31e4ef2fa7e5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:51:59.560973 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:51:59.633335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82944277-f92f-4fd8-95f8-31e4ef2fa7e5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:51:59.666883 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c99ea-8fd6-4ff1-ba61-8330ebabb7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:51:59.667521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c99ea-8fd6-4ff1-ba61-8330ebabb7ed None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:51:59.667800 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c99ea-8fd6-4ff1-ba61-8330ebabb7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:51:59.668072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c99ea-8fd6-4ff1-ba61-8330ebabb7ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:51:59.668918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c99ea-8fd6-4ff1-ba61-8330ebabb7ed None admin] RBAC: Adding query filter params (role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb, scope.project.id=155180ba63c446ca889a6eb3f94cc31a, user.id=9715d5deeccd419b97cb0bd39e17982f) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:51:59.669225 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c99ea-8fd6-4ff1-ba61-8330ebabb7ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:51:59.670651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c99ea-8fd6-4ff1-ba61-8330ebabb7ed None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:51:59.675973 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 625/1260] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:51:59 2022] GET /identity/v3/role_assignments?user.id=9715d5deeccd419b97cb0bd39e17982f&scope.project.id=155180ba63c446ca889a6eb3f94cc31a&role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:52:01.664319 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85be2b9-1f13-4c90-aea1-d24a6faadb98 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:01.664868 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85be2b9-1f13-4c90-aea1-d24a6faadb98 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:01.664976 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85be2b9-1f13-4c90-aea1-d24a6faadb98 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:01.666255 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 636/1261] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:52:01.672055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409d97b-d46f-4f55-ae22-30db2d090e26 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:01.672514 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409d97b-d46f-4f55-ae22-30db2d090e26 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:01.672636 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409d97b-d46f-4f55-ae22-30db2d090e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:01.690106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b409d97b-d46f-4f55-ae22-30db2d090e26 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:52:01.691327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b409d97b-d46f-4f55-ae22-30db2d090e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:52:01.757564 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 626/1262] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:52:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:52:01.780590 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85be2b9-1f13-4c90-aea1-d24a6faadb98 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:01.791213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:52:01.878886 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85be2b9-1f13-4c90-aea1-d24a6faadb98 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:01.913184 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9897b63-5958-4016-81b2-83efc8c65a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:01.914064 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9897b63-5958-4016-81b2-83efc8c65a78 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:01.918077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9897b63-5958-4016-81b2-83efc8c65a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:01.918077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9897b63-5958-4016-81b2-83efc8c65a78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:01.925189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9897b63-5958-4016-81b2-83efc8c65a78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:52:01.932630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-c9897b63-5958-4016-81b2-83efc8c65a78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:52:01.932630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:52:01.932630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:52:01.932630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:52:01.932630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:52:01.932630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:52:01.932630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:52:01.932630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:52:01.932630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:52:01.932630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:52:01.932630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:52:01.932630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:52:01.932630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:52:01.932630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:52:01.932630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:52:01.932630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:52:01.933777 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:52:01.933777 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:52:01.933777 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:52:01.933777 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:52:01.933777 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:52:01.933777 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:52:01.933777 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:52:01.933777 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:52:01.933777 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:52:01.933777 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:52:01.933777 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:52:01.933777 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:52:01.933777 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:52:01.933777 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:52:01.933777 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:52:01.933777 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:52:01.934666 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:52:01.934666 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:52:01.934666 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:52:01.934666 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:52:01.934666 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:52:01.934666 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:52:01.934666 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:52:01.934666 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:52:01.934666 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:52:01.934666 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:52:01.934666 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:52:01.934666 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:52:01.934666 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 637/1263] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:52:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:52:01.940176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b409d97b-d46f-4f55-ae22-30db2d090e26 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:01.957958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:52:02.034741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b409d97b-d46f-4f55-ae22-30db2d090e26 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:02.068153 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698edc05-e381-4733-a613-ffd50d5c2787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:02.068836 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698edc05-e381-4733-a613-ffd50d5c2787 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:02.069117 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698edc05-e381-4733-a613-ffd50d5c2787 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:02.069391 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698edc05-e381-4733-a613-ffd50d5c2787 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:02.070131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698edc05-e381-4733-a613-ffd50d5c2787 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:52:02.070434 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698edc05-e381-4733-a613-ffd50d5c2787 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:02.071790 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698edc05-e381-4733-a613-ffd50d5c2787 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:02.079766 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 627/1264] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:52:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:52:02.085411 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9897b63-5958-4016-81b2-83efc8c65a78 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:02.150449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:52:02.183125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9897b63-5958-4016-81b2-83efc8c65a78 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:02.215052 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b3fabf-3491-424d-97ef-063764c51f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:02.215828 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3fabf-3491-424d-97ef-063764c51f93 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:02.216332 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3fabf-3491-424d-97ef-063764c51f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:02.216458 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3fabf-3491-424d-97ef-063764c51f93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:02.228373 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b3fabf-3491-424d-97ef-063764c51f93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:02.229835 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b3fabf-3491-424d-97ef-063764c51f93 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:02.246802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-c7b3fabf-3491-424d-97ef-063764c51f93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:52:02.246802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:52:02.246802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:52:02.246802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:52:02.246802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:52:02.246802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:52:02.246802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:52:02.246802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:52:02.246802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:52:02.246802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:52:02.246802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:52:02.246802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:52:02.246802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:52:02.246802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:52:02.246802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:52:02.247830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:52:02.247830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:52:02.247830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:52:02.247830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:52:02.247830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:52:02.247830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:52:02.247830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:52:02.247830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:52:02.247830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:52:02.247830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:52:02.247830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:52:02.247830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:52:02.247830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:52:02.247830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:52:02.247830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:52:02.247830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:52:02.247830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:52:02.248971 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:52:02.248971 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:52:02.248971 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:52:02.248971 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:52:02.248971 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 638/1265] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:52:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:52:02.253754 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698edc05-e381-4733-a613-ffd50d5c2787 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:02.335162 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:52:02.353117 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698edc05-e381-4733-a613-ffd50d5c2787 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:02.386618 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba8287-bcc5-4a8a-a923-11c0d73ef726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:02.387313 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba8287-bcc5-4a8a-a923-11c0d73ef726 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:02.387595 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba8287-bcc5-4a8a-a923-11c0d73ef726 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:02.387874 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba8287-bcc5-4a8a-a923-11c0d73ef726 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:02.388616 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba8287-bcc5-4a8a-a923-11c0d73ef726 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:52:02.388925 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba8287-bcc5-4a8a-a923-11c0d73ef726 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:02.390331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba8287-bcc5-4a8a-a923-11c0d73ef726 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:02.401298 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 628/1266] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:52:02.406859 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b3fabf-3491-424d-97ef-063764c51f93 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:02.509091 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b3fabf-3491-424d-97ef-063764c51f93 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:02.528497 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:52:02.542170 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b0884-a9da-4adf-a2d0-5e8bc003f873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:02.542997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b0884-a9da-4adf-a2d0-5e8bc003f873 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:02.543279 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b0884-a9da-4adf-a2d0-5e8bc003f873 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:02.543549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b0884-a9da-4adf-a2d0-5e8bc003f873 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:02.550702 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b0884-a9da-4adf-a2d0-5e8bc003f873 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:02.552135 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b0884-a9da-4adf-a2d0-5e8bc003f873 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:02.566312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-bf8b0884-a9da-4adf-a2d0-5e8bc003f873 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 19:52:02.566312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:52:02.566312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:52:02.566312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:52:02.566312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:52:02.566312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:52:02.566312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:52:02.566312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:52:02.566312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:52:02.566312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:52:02.566312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:52:02.566312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:52:02.566312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:52:02.566312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:52:02.566312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:52:02.566312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:52:02.571119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:52:02.571119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:52:02.571119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:52:02.571119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:52:02.571119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:52:02.571119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:52:02.571119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:52:02.571119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:52:02.571119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:52:02.571119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:52:02.571119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:52:02.571119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:52:02.571119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:52:02.571119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:52:02.571119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:52:02.571119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:52:02.571119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:52:02.571119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:52:02.572403 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:52:02.572403 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:52:02.572403 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:52:02.572403 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:52:02.572403 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:52:02.572403 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:52:02.572403 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:52:02.572403 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:52:02.572403 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 19:52:02.572403 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:52:02.572403 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 639/1267] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:52:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:52:02.573558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba8287-bcc5-4a8a-a923-11c0d73ef726 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:02.671162 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba8287-bcc5-4a8a-a923-11c0d73ef726 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:02.704523 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1f4e2-641f-416b-ae1c-4bc10edd3fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:02.705201 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1f4e2-641f-416b-ae1c-4bc10edd3fdf None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:02.705463 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1f4e2-641f-416b-ae1c-4bc10edd3fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:02.705738 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1f4e2-641f-416b-ae1c-4bc10edd3fdf None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:02.706565 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1f4e2-641f-416b-ae1c-4bc10edd3fdf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:52:02.706872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1f4e2-641f-416b-ae1c-4bc10edd3fdf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:02.708310 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1f4e2-641f-416b-ae1c-4bc10edd3fdf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:02.717362 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:52:02.725194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 629/1268] 10.222.0.44 () {56 vars in 1143 bytes} [Thu Jun 23 19:52:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:52:02.730774 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b0884-a9da-4adf-a2d0-5e8bc003f873 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:02.830061 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b0884-a9da-4adf-a2d0-5e8bc003f873 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:02.862258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e3352-8edb-4c63-9dc8-238d35dd756d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:02.863037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e3352-8edb-4c63-9dc8-238d35dd756d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:02.863319 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e3352-8edb-4c63-9dc8-238d35dd756d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:02.863593 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e3352-8edb-4c63-9dc8-238d35dd756d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:02.874876 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e3352-8edb-4c63-9dc8-238d35dd756d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:02.876344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e3352-8edb-4c63-9dc8-238d35dd756d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:02.893493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-6d0e3352-8edb-4c63-9dc8-238d35dd756d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:52:02.893493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:52:02.893493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:52:02.893493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:52:02.893493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:52:02.893493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:52:02.893493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:52:02.893493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:52:02.893493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:52:02.893493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:52:02.893493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:52:02.893493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:52:02.893493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:52:02.893493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:52:02.893493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:52:02.894590 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:52:02.894590 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:52:02.894590 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:52:02.894590 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:52:02.894590 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:52:02.894590 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:52:02.894590 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:52:02.894590 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:52:02.894590 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:52:02.894590 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:52:02.894590 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:52:02.894590 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:52:02.894590 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:52:02.894590 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:52:02.894590 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:52:02.894590 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:52:02.894590 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:52:02.895697 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:52:02.895697 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:52:02.895697 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:52:02.895697 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:52:02.895697 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 640/1269] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:52:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:52:02.900412 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1f4e2-641f-416b-ae1c-4bc10edd3fdf None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:03.000089 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1f4e2-641f-416b-ae1c-4bc10edd3fdf None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:03.033933 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1941693-47c4-4e46-8cea-3c28683fd258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:03.034578 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1941693-47c4-4e46-8cea-3c28683fd258 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:03.034858 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1941693-47c4-4e46-8cea-3c28683fd258 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:03.035127 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1941693-47c4-4e46-8cea-3c28683fd258 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:03.035830 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1941693-47c4-4e46-8cea-3c28683fd258 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:52:03.036132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1941693-47c4-4e46-8cea-3c28683fd258 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:03.037520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1941693-47c4-4e46-8cea-3c28683fd258 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:03.048951 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 630/1270] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:52:03.054514 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e3352-8edb-4c63-9dc8-238d35dd756d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:03.089281 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:52:03.151187 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e3352-8edb-4c63-9dc8-238d35dd756d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:03.183047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec88e7-ea77-4455-b605-7e928a320d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:03.184020 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec88e7-ea77-4455-b605-7e928a320d01 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:03.184381 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec88e7-ea77-4455-b605-7e928a320d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:03.184654 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec88e7-ea77-4455-b605-7e928a320d01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:03.192902 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ec88e7-ea77-4455-b605-7e928a320d01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:03.194345 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ec88e7-ea77-4455-b605-7e928a320d01 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:03.208178 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-67ec88e7-ea77-4455-b605-7e928a320d01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:52:03.208178 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:52:03.208178 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:52:03.208178 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:52:03.208178 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:52:03.208178 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:52:03.208178 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:52:03.208178 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:52:03.208178 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:52:03.208178 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:52:03.208178 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:52:03.208178 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:52:03.208178 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:52:03.208178 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:52:03.208178 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:52:03.209158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:52:03.209158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:52:03.209158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:52:03.209158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:52:03.209158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:52:03.209158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:52:03.209158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:52:03.209158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:52:03.209158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:52:03.209158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:52:03.209158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:52:03.209158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:52:03.209158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:52:03.209158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:52:03.209158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:52:03.209158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:52:03.209158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:52:03.210410 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:52:03.210410 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:52:03.210410 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:52:03.210410 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:52:03.210410 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:52:03.210410 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:52:03.210410 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:52:03.210410 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:52:03.210410 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 641/1271] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:52:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:52:03.215171 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1941693-47c4-4e46-8cea-3c28683fd258 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:03.273313 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:52:03.323040 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1941693-47c4-4e46-8cea-3c28683fd258 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:03.356898 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79e999-da37-4997-becd-7edd3d7b3c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:03.358078 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79e999-da37-4997-becd-7edd3d7b3c2b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:03.358567 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79e999-da37-4997-becd-7edd3d7b3c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:03.358871 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79e999-da37-4997-becd-7edd3d7b3c2b None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:03.359686 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd79e999-da37-4997-becd-7edd3d7b3c2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:52:03.359994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd79e999-da37-4997-becd-7edd3d7b3c2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:03.361469 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd79e999-da37-4997-becd-7edd3d7b3c2b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:03.373128 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 631/1272] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:52:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:52:03.379082 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec88e7-ea77-4455-b605-7e928a320d01 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:03.467361 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:52:03.475860 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec88e7-ea77-4455-b605-7e928a320d01 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:03.508744 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb3617-e0c4-4e0c-b2b7-4d7d4c3e330c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:03.509624 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb3617-e0c4-4e0c-b2b7-4d7d4c3e330c None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:03.509934 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb3617-e0c4-4e0c-b2b7-4d7d4c3e330c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:03.510226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb3617-e0c4-4e0c-b2b7-4d7d4c3e330c None admin] PATH_INFO: `/v3/projects/155180ba63c446ca889a6eb3f94cc31a/users/9715d5deeccd419b97cb0bd39e17982f/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:03.512236 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb3617-e0c4-4e0c-b2b7-4d7d4c3e330c None admin] RBAC: Authorizing `identity:create_grant(project_id=155180ba63c446ca889a6eb3f94cc31a, user_id=9715d5deeccd419b97cb0bd39e17982f, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:03.513696 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb3617-e0c4-4e0c-b2b7-4d7d4c3e330c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:03.599803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 642/1273] 10.222.0.44 () {58 vars in 1439 bytes} [Thu Jun 23 19:52:03 2022] PUT /identity/v3/projects/155180ba63c446ca889a6eb3f94cc31a/users/9715d5deeccd419b97cb0bd39e17982f/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 19:52:05.629825 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19255573-49e2-4276-92ef-629924d9938a None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:05.630359 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19255573-49e2-4276-92ef-629924d9938a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:05.630359 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19255573-49e2-4276-92ef-629924d9938a None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:05.631745 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 632/1274] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:52:05.637167 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07e611-a6cf-450d-b1fa-0e45c106e1c4 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:05.637609 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07e611-a6cf-450d-b1fa-0e45c106e1c4 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:05.637712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07e611-a6cf-450d-b1fa-0e45c106e1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:05.654393 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-1c07e611-a6cf-450d-b1fa-0e45c106e1c4 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:52:05.655576 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1c07e611-a6cf-450d-b1fa-0e45c106e1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:52:05.728456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 643/1275] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:52:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:52:05.753077 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19255573-49e2-4276-92ef-629924d9938a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:05.851071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19255573-49e2-4276-92ef-629924d9938a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:05.867399 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:52:05.883569 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49b9dc-fbf1-44a6-8493-0f193b1ca60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:05.884289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49b9dc-fbf1-44a6-8493-0f193b1ca60f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:05.884598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49b9dc-fbf1-44a6-8493-0f193b1ca60f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:05.884920 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab49b9dc-fbf1-44a6-8493-0f193b1ca60f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:05.894998 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab49b9dc-fbf1-44a6-8493-0f193b1ca60f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 19:52:05.902473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-ab49b9dc-fbf1-44a6-8493-0f193b1ca60f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:52:05.902473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:52:05.902473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:52:05.902473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:52:05.902473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:52:05.902473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:52:05.902473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:52:05.902473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:52:05.902473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:52:05.902473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:52:05.902473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:52:05.902473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:52:05.902473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 19:52:05.902473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 19:52:05.902473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 19:52:05.902473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 19:52:05.903633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 19:52:05.903633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 19:52:05.903633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 19:52:05.903633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 19:52:05.903633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:52:05.903633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:52:05.903633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:52:05.903633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:52:05.903633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:52:05.903633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:52:05.903633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:52:05.903633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:52:05.903633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:52:05.903633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:52:05.903633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:52:05.903633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:52:05.904714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:52:05.904714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:52:05.904714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:52:05.904714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:52:05.904714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 19:52:05.904714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 19:52:05.904714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 19:52:05.904714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 19:52:05.904714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 19:52:05.904714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 19:52:05.904714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 19:52:05.904714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:52:05.904714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 633/1276] 10.222.0.44 () {56 vars in 1096 bytes} [Thu Jun 23 19:52:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:52:05.909888 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07e611-a6cf-450d-b1fa-0e45c106e1c4 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:05.990201 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:52:06.007817 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07e611-a6cf-450d-b1fa-0e45c106e1c4 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:06.039284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42d6f1-985a-49b2-9d07-745eeb82d967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:06.039950 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42d6f1-985a-49b2-9d07-745eeb82d967 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:06.040235 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42d6f1-985a-49b2-9d07-745eeb82d967 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:06.040577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42d6f1-985a-49b2-9d07-745eeb82d967 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:06.041327 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a42d6f1-985a-49b2-9d07-745eeb82d967 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:52:06.041636 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a42d6f1-985a-49b2-9d07-745eeb82d967 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:06.043016 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a42d6f1-985a-49b2-9d07-745eeb82d967 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:06.051205 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 644/1277] 10.222.0.44 () {56 vars in 1099 bytes} [Thu Jun 23 19:52:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:52:06.056970 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49b9dc-fbf1-44a6-8493-0f193b1ca60f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:06.159715 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab49b9dc-fbf1-44a6-8493-0f193b1ca60f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:06.192828 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a21ff-2d1d-4ff8-a8cf-28af29c42ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:06.193656 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a21ff-2d1d-4ff8-a8cf-28af29c42ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:06.193932 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a21ff-2d1d-4ff8-a8cf-28af29c42ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:06.194195 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a21ff-2d1d-4ff8-a8cf-28af29c42ad4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:06.204437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6a21ff-2d1d-4ff8-a8cf-28af29c42ad4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:06.205979 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6a21ff-2d1d-4ff8-a8cf-28af29c42ad4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:06.222749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-dc6a21ff-2d1d-4ff8-a8cf-28af29c42ad4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:52:06.222749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:52:06.222749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:52:06.222749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:52:06.222749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:52:06.222749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:52:06.222749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:52:06.222749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:52:06.222749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:52:06.222749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:52:06.222749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:52:06.222749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:52:06.222749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:52:06.222749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:52:06.222749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:52:06.223841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:52:06.223841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:52:06.223841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:52:06.223841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:52:06.223841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:52:06.223841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:52:06.223841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:52:06.223841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:52:06.223841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:52:06.223841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:52:06.223841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:52:06.223841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:52:06.223841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:52:06.223841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:52:06.223841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:52:06.223841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:52:06.223841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:52:06.224831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:52:06.224831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:52:06.224831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:52:06.224831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:52:06.224831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 634/1278] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:52:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:52:06.229874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42d6f1-985a-49b2-9d07-745eeb82d967 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:06.329244 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42d6f1-985a-49b2-9d07-745eeb82d967 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:06.361684 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384a72c-4301-4430-97e5-d67e9e18ea2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:06.362330 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384a72c-4301-4430-97e5-d67e9e18ea2d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:06.362607 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384a72c-4301-4430-97e5-d67e9e18ea2d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:06.362878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384a72c-4301-4430-97e5-d67e9e18ea2d None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:06.363581 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384a72c-4301-4430-97e5-d67e9e18ea2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:52:06.363880 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384a72c-4301-4430-97e5-d67e9e18ea2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:06.365280 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384a72c-4301-4430-97e5-d67e9e18ea2d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:06.376615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 645/1279] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:52:06.382332 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a21ff-2d1d-4ff8-a8cf-28af29c42ad4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:06.413105 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:52:06.478574 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a21ff-2d1d-4ff8-a8cf-28af29c42ad4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:06.511991 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd01421-5a4b-4a79-b722-17ba9722d856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:06.512813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd01421-5a4b-4a79-b722-17ba9722d856 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:06.513091 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd01421-5a4b-4a79-b722-17ba9722d856 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:06.513362 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd01421-5a4b-4a79-b722-17ba9722d856 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:06.520381 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd01421-5a4b-4a79-b722-17ba9722d856 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:06.521834 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd01421-5a4b-4a79-b722-17ba9722d856 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:06.536537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-acd01421-5a4b-4a79-b722-17ba9722d856 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 19:52:06.536537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:52:06.536537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:52:06.536537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:52:06.536537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:52:06.536537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:52:06.536537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:52:06.536537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:52:06.536537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:52:06.536537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:52:06.536537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:52:06.536537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:52:06.536537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 19:52:06.536537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 19:52:06.536537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 19:52:06.536537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 19:52:06.537499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:52:06.537499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:52:06.537499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 19:52:06.537499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:52:06.537499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 19:52:06.537499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 19:52:06.537499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:52:06.537499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:52:06.537499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:52:06.537499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:52:06.537499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:52:06.537499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:52:06.537499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:52:06.537499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:52:06.537499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:52:06.537499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:52:06.537499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:52:06.537499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:52:06.538509 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:52:06.538509 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:52:06.538509 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 19:52:06.538509 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 19:52:06.538509 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 19:52:06.538509 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 19:52:06.538509 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 19:52:06.538509 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 19:52:06.538509 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 19:52:06.538509 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:52:06.538509 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 635/1280] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:52:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:52:06.543553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384a72c-4301-4430-97e5-d67e9e18ea2d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:06.547814 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:52:06.644709 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384a72c-4301-4430-97e5-d67e9e18ea2d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:06.678210 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d889db2c-a6ae-41f9-9226-292fbc0156a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:06.678816 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889db2c-a6ae-41f9-9226-292fbc0156a8 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:06.679136 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889db2c-a6ae-41f9-9226-292fbc0156a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:06.679414 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889db2c-a6ae-41f9-9226-292fbc0156a8 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:06.680249 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d889db2c-a6ae-41f9-9226-292fbc0156a8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:52:06.680589 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d889db2c-a6ae-41f9-9226-292fbc0156a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:06.682020 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d889db2c-a6ae-41f9-9226-292fbc0156a8 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:06.698744 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 646/1281] 10.222.0.44 () {56 vars in 1143 bytes} [Thu Jun 23 19:52:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:52:06.705223 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd01421-5a4b-4a79-b722-17ba9722d856 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:06.792829 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:52:06.802098 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd01421-5a4b-4a79-b722-17ba9722d856 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:06.835622 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2090a-148d-4620-accf-302c246f8572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:06.836252 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2090a-148d-4620-accf-302c246f8572 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:06.836560 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2090a-148d-4620-accf-302c246f8572 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:06.836829 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2090a-148d-4620-accf-302c246f8572 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:06.900102 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa2090a-148d-4620-accf-302c246f8572 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:06.901572 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa2090a-148d-4620-accf-302c246f8572 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:06.988588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:52:06.998541 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 636/1282] 10.222.0.44 () {58 vars in 1282 bytes} [Thu Jun 23 19:52:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 19:52:07.004667 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d889db2c-a6ae-41f9-9226-292fbc0156a8 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:07.064669 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:52:07.102068 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d889db2c-a6ae-41f9-9226-292fbc0156a8 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:07.134668 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f0fe5-a372-487f-8b40-de9f3f0a330d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:07.136795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f0fe5-a372-487f-8b40-de9f3f0a330d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:07.136795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f0fe5-a372-487f-8b40-de9f3f0a330d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:07.136795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f0fe5-a372-487f-8b40-de9f3f0a330d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:07.146113 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f0fe5-a372-487f-8b40-de9f3f0a330d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:07.147612 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f0fe5-a372-487f-8b40-de9f3f0a330d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:07.164440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-310f0fe5-a372-487f-8b40-de9f3f0a330d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:52:07.164440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:52:07.164440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:52:07.164440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:52:07.164440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:52:07.164440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:52:07.164440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:52:07.164440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:52:07.164440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:52:07.164440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:52:07.164440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:52:07.164440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:52:07.164440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 19:52:07.164440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 19:52:07.164440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 19:52:07.165466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 19:52:07.165466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:52:07.165466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:52:07.165466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:52:07.165466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:52:07.165466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:52:07.165466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:52:07.165466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:52:07.165466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:52:07.165466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:52:07.165466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:52:07.165466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:52:07.165466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:52:07.165466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:52:07.165466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:52:07.165466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:52:07.165466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:52:07.166587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 19:52:07.166587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 19:52:07.166587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 19:52:07.166587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:52:07.166587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 647/1283] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 19:52:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:52:07.171431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2090a-148d-4620-accf-302c246f8572 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:07.269518 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2090a-148d-4620-accf-302c246f8572 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:07.301960 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116130a5-3695-4dcd-ad3f-61c07973656c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:07.309024 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116130a5-3695-4dcd-ad3f-61c07973656c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:07.309024 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116130a5-3695-4dcd-ad3f-61c07973656c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:07.309024 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116130a5-3695-4dcd-ad3f-61c07973656c None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:07.309024 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116130a5-3695-4dcd-ad3f-61c07973656c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:52:07.309024 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116130a5-3695-4dcd-ad3f-61c07973656c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:07.309024 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116130a5-3695-4dcd-ad3f-61c07973656c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:07.317113 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 637/1284] 10.222.0.44 () {56 vars in 1109 bytes} [Thu Jun 23 19:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:52:07.322856 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f0fe5-a372-487f-8b40-de9f3f0a330d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:07.423598 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f0fe5-a372-487f-8b40-de9f3f0a330d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:07.437776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:52:07.457502 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050e15c-7b64-40ce-86bf-98a85ddc5f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:07.458283 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050e15c-7b64-40ce-86bf-98a85ddc5f0b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:07.458562 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050e15c-7b64-40ce-86bf-98a85ddc5f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:07.458832 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050e15c-7b64-40ce-86bf-98a85ddc5f0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:07.465180 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050e15c-7b64-40ce-86bf-98a85ddc5f0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:07.466614 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050e15c-7b64-40ce-86bf-98a85ddc5f0b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:07.480318 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-e050e15c-7b64-40ce-86bf-98a85ddc5f0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:52:07.480318 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:52:07.480318 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:52:07.480318 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:52:07.480318 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:52:07.480318 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:52:07.480318 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:52:07.480318 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:52:07.480318 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:52:07.480318 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:52:07.480318 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:52:07.480318 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:52:07.480318 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 19:52:07.480318 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 19:52:07.480318 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 19:52:07.481349 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 19:52:07.481349 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:52:07.481349 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:52:07.481349 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:52:07.481349 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:52:07.481349 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:52:07.481349 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:52:07.481349 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:52:07.481349 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jun 23 19:52:07.481349 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:52:07.481349 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:52:07.481349 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:52:07.481349 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:52:07.481349 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:52:07.481349 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:52:07.481349 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:52:07.481349 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:52:07.482916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 19:52:07.482916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 19:52:07.482916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 19:52:07.482916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 19:52:07.482916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 19:52:07.482916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 19:52:07.482916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 19:52:07.482916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jun 23 19:52:07.482916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 648/1285] 10.222.0.44 () {56 vars in 1111 bytes} [Thu Jun 23 19:52:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:52:07.487875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116130a5-3695-4dcd-ad3f-61c07973656c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:07.540966 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:52:07.589122 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116130a5-3695-4dcd-ad3f-61c07973656c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:07.621436 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ca692-5570-4ca5-b2c3-a43bd7b7b87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:07.622089 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ca692-5570-4ca5-b2c3-a43bd7b7b87a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:07.622364 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ca692-5570-4ca5-b2c3-a43bd7b7b87a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:07.622638 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ca692-5570-4ca5-b2c3-a43bd7b7b87a None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:07.624041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ca692-5570-4ca5-b2c3-a43bd7b7b87a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:52:07.624374 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ca692-5570-4ca5-b2c3-a43bd7b7b87a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:07.625805 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ca692-5570-4ca5-b2c3-a43bd7b7b87a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:07.637311 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 638/1286] 10.222.0.44 () {56 vars in 1148 bytes} [Thu Jun 23 19:52:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 19:52:07.643378 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050e15c-7b64-40ce-86bf-98a85ddc5f0b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:07.741754 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050e15c-7b64-40ce-86bf-98a85ddc5f0b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:07.775701 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f7b359-e4d4-46ab-8c25-ca802de8ef4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:07.776397 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7b359-e4d4-46ab-8c25-ca802de8ef4a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:07.776678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7b359-e4d4-46ab-8c25-ca802de8ef4a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:07.776945 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f7b359-e4d4-46ab-8c25-ca802de8ef4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:07.777759 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f7b359-e4d4-46ab-8c25-ca802de8ef4a None admin] RBAC: Adding query filter params (role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb, scope.project.id=155180ba63c446ca889a6eb3f94cc31a, user.id=9715d5deeccd419b97cb0bd39e17982f) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:52:07.778064 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f7b359-e4d4-46ab-8c25-ca802de8ef4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:07.779528 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f7b359-e4d4-46ab-8c25-ca802de8ef4a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:07.785160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 649/1287] 10.222.0.44 () {56 vars in 1374 bytes} [Thu Jun 23 19:52:07 2022] GET /identity/v3/role_assignments?user.id=9715d5deeccd419b97cb0bd39e17982f&scope.project.id=155180ba63c446ca889a6eb3f94cc31a&role.id=a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:52:09.845993 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef0adc-d81e-4c4b-b34c-3c9bf6a88287 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:09.846547 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef0adc-d81e-4c4b-b34c-3c9bf6a88287 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:09.846547 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef0adc-d81e-4c4b-b34c-3c9bf6a88287 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:09.847909 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 639/1288] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:52:09.853159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7098a-9bb5-4561-b428-5407eb2e9fcc None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:09.853599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7098a-9bb5-4561-b428-5407eb2e9fcc None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:09.853709 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7098a-9bb5-4561-b428-5407eb2e9fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:09.870140 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d1c7098a-9bb5-4561-b428-5407eb2e9fcc None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:52:09.871320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d1c7098a-9bb5-4561-b428-5407eb2e9fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:52:09.900435 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:52:09.936695 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 650/1289] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:52:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:52:09.960028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef0adc-d81e-4c4b-b34c-3c9bf6a88287 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:10.058859 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef0adc-d81e-4c4b-b34c-3c9bf6a88287 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:10.069843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:52:10.093396 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d6e4a-1bd6-4d41-a996-edf04daab14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:10.094194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d6e4a-1bd6-4d41-a996-edf04daab14a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:10.094477 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d6e4a-1bd6-4d41-a996-edf04daab14a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:10.094751 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d6e4a-1bd6-4d41-a996-edf04daab14a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:10.095400 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d6e4a-1bd6-4d41-a996-edf04daab14a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:10.096867 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d6e4a-1bd6-4d41-a996-edf04daab14a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:10.108346 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-387d6e4a-1bd6-4d41-a996-edf04daab14a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 19:52:10.108346 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:52:10.108346 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 19:52:10.108346 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 19:52:10.108346 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 19:52:10.108346 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 19:52:10.108346 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 19:52:10.108346 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:52:10.108346 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 19:52:10.108346 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:52:10.108346 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:52:10.108346 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:52:10.108346 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:52:10.108346 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:52:10.108346 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:52:10.108346 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:52:10.108346 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:52:10.118174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:52:10.118174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:52:10.118174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:52:10.118174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:52:10.118174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:52:10.118174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:52:10.118174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 19:52:10.118174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 19:52:10.118174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 19:52:10.118174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 19:52:10.118174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:52:10.118174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:52:10.118174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:52:10.118174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:52:10.118174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:52:10.118174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:52:10.119034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:52:10.119034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:52:10.119034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:52:10.119034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:52:10.119034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:52:10.119034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:52:10.119034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:52:10.119034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:52:10.119034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:52:10.119034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:52:10.119034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 19:52:10.119034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:52:10.119034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 19:52:10.119034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:52:10.119034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 640/1290] 10.222.0.44 () {56 vars in 1117 bytes} [Thu Jun 23 19:52:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:52:10.119847 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c7098a-9bb5-4561-b428-5407eb2e9fcc None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:10.215269 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c7098a-9bb5-4561-b428-5407eb2e9fcc None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:10.251979 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f27354-5dad-493e-8a94-4fa1e191e3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:10.252667 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f27354-5dad-493e-8a94-4fa1e191e3c5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:10.252946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f27354-5dad-493e-8a94-4fa1e191e3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:10.253216 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f27354-5dad-493e-8a94-4fa1e191e3c5 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:10.253917 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f27354-5dad-493e-8a94-4fa1e191e3c5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:52:10.254216 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f27354-5dad-493e-8a94-4fa1e191e3c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:10.255582 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f27354-5dad-493e-8a94-4fa1e191e3c5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:10.260378 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 651/1291] 10.222.0.44 () {56 vars in 1116 bytes} [Thu Jun 23 19:52:10 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:52:10.266826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d6e4a-1bd6-4d41-a996-edf04daab14a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:10.363467 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d6e4a-1bd6-4d41-a996-edf04daab14a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:10.396565 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc233dd-5bef-49e8-b102-cf6eda1d565f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:10.397212 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc233dd-5bef-49e8-b102-cf6eda1d565f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:10.397496 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc233dd-5bef-49e8-b102-cf6eda1d565f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:10.397771 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc233dd-5bef-49e8-b102-cf6eda1d565f None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:10.398466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc233dd-5bef-49e8-b102-cf6eda1d565f None admin] RBAC: Adding query filter params (type=placement) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:52:10.398773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc233dd-5bef-49e8-b102-cf6eda1d565f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:10.400153 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc233dd-5bef-49e8-b102-cf6eda1d565f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:10.404607 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 641/1292] 10.222.0.44 () {56 vars in 1116 bytes} [Thu Jun 23 19:52:10 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:52:12.441506 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ce99f-c17d-4bb5-8c2d-f6bea5978265 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:12.442078 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ce99f-c17d-4bb5-8c2d-f6bea5978265 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:12.442078 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ce99f-c17d-4bb5-8c2d-f6bea5978265 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:12.443404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 652/1293] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:52:12.448593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9f113-395b-45d8-a768-754e2a9c48c8 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:12.449016 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9f113-395b-45d8-a768-754e2a9c48c8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:12.449153 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9f113-395b-45d8-a768-754e2a9c48c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:12.466219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9fa9f113-395b-45d8-a768-754e2a9c48c8 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:52:12.470628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9fa9f113-395b-45d8-a768-754e2a9c48c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:52:12.495737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:52:12.531991 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 642/1294] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:52:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:52:12.554821 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ce99f-c17d-4bb5-8c2d-f6bea5978265 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:12.585848 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:52:12.653819 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ce99f-c17d-4bb5-8c2d-f6bea5978265 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:12.687385 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673dc96e-c2cb-4006-b67e-857fb4e6798d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:12.688290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dc96e-c2cb-4006-b67e-857fb4e6798d None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:12.688596 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dc96e-c2cb-4006-b67e-857fb4e6798d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:12.688867 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dc96e-c2cb-4006-b67e-857fb4e6798d None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:12.689574 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673dc96e-c2cb-4006-b67e-857fb4e6798d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:12.690985 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673dc96e-c2cb-4006-b67e-857fb4e6798d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:12.841019 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 653/1295] 10.222.0.44 () {60 vars in 1141 bytes} [Thu Jun 23 19:52:12 2022] POST /identity/v3/services => generated 246 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:52:14.850169 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbebc56-adb8-466a-a8e2-61b2f7508d0c None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:14.850800 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbebc56-adb8-466a-a8e2-61b2f7508d0c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:14.850800 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbebc56-adb8-466a-a8e2-61b2f7508d0c None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:14.852132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 643/1296] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:52:14.857218 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5eee65-50bb-4139-94d1-32cb89a082be None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:14.858144 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5eee65-50bb-4139-94d1-32cb89a082be None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:14.858315 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5eee65-50bb-4139-94d1-32cb89a082be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:14.874802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8c5eee65-50bb-4139-94d1-32cb89a082be None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:52:14.875974 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8c5eee65-50bb-4139-94d1-32cb89a082be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:52:14.947008 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 654/1297] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:52:14 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:52:14.970357 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbebc56-adb8-466a-a8e2-61b2f7508d0c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:15.068217 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbebc56-adb8-466a-a8e2-61b2f7508d0c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:15.101728 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cca24-d809-4ba0-be7e-c6fc687480c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:15.102454 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cca24-d809-4ba0-be7e-c6fc687480c2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:15.102737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cca24-d809-4ba0-be7e-c6fc687480c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:15.103009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cca24-d809-4ba0-be7e-c6fc687480c2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:15.103651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4cca24-d809-4ba0-be7e-c6fc687480c2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:15.105045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4cca24-d809-4ba0-be7e-c6fc687480c2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:15.116774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-7d4cca24-d809-4ba0-be7e-c6fc687480c2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 19:52:15.116774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:52:15.116774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 19:52:15.116774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 19:52:15.116774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 19:52:15.116774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 19:52:15.116774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 19:52:15.116774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:52:15.116774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 19:52:15.116774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:52:15.116774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:52:15.116774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:52:15.116774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:52:15.116774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:52:15.116774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:52:15.116774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:52:15.116774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:52:15.118041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:52:15.118041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:52:15.118041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:52:15.118041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:52:15.118041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:52:15.118041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:52:15.118041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 19:52:15.118041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 19:52:15.118041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 19:52:15.118041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 19:52:15.118041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:52:15.118041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:52:15.118041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:52:15.118041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:52:15.118041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:52:15.118041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:52:15.118979 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:52:15.118979 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:52:15.118979 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:52:15.118979 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:52:15.118979 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:52:15.118979 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:52:15.118979 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:52:15.118979 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:52:15.118979 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:52:15.118979 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:52:15.118979 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 19:52:15.118979 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:52:15.118979 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 19:52:15.118979 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:52:15.118979 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 644/1298] 10.222.0.44 () {56 vars in 1117 bytes} [Thu Jun 23 19:52:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:52:15.123891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5eee65-50bb-4139-94d1-32cb89a082be None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:15.142310 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:52:15.220580 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5eee65-50bb-4139-94d1-32cb89a082be None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:15.254361 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0413ad4-d408-4089-b0e6-53ac917397f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:15.255017 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0413ad4-d408-4089-b0e6-53ac917397f2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:15.255296 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0413ad4-d408-4089-b0e6-53ac917397f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:15.255562 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0413ad4-d408-4089-b0e6-53ac917397f2 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:15.256267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0413ad4-d408-4089-b0e6-53ac917397f2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:52:15.256590 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0413ad4-d408-4089-b0e6-53ac917397f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:15.258994 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0413ad4-d408-4089-b0e6-53ac917397f2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:15.262972 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 655/1299] 10.222.0.44 () {56 vars in 1116 bytes} [Thu Jun 23 19:52:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:52:15.269503 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cca24-d809-4ba0-be7e-c6fc687480c2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:15.300908 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:52:15.368838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cca24-d809-4ba0-be7e-c6fc687480c2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:15.409529 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5c9ec-ef64-4308-abb5-00c5d869629b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:15.410186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5c9ec-ef64-4308-abb5-00c5d869629b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:15.410466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5c9ec-ef64-4308-abb5-00c5d869629b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:15.410738 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5c9ec-ef64-4308-abb5-00c5d869629b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:15.411517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5c9ec-ef64-4308-abb5-00c5d869629b None admin] RBAC: Adding query filter params (interface=public, service_id=330da418306a40aeb3b044a285ee7883, region_id=RegionOne) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:52:15.411826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5c9ec-ef64-4308-abb5-00c5d869629b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:15.413217 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5c9ec-ef64-4308-abb5-00c5d869629b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:15.418237 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 645/1300] 10.222.0.44 () {56 vars in 1251 bytes} [Thu Jun 23 19:52:15 2022] GET /identity/v3/endpoints?service_id=330da418306a40aeb3b044a285ee7883&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:52:15.424241 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0413ad4-d408-4089-b0e6-53ac917397f2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:15.503500 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:52:15.521217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0413ad4-d408-4089-b0e6-53ac917397f2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:15.553832 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd25ac-e79a-42d8-ac15-bb8b753de996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:15.554481 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd25ac-e79a-42d8-ac15-bb8b753de996 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:15.554762 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd25ac-e79a-42d8-ac15-bb8b753de996 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:15.555035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd25ac-e79a-42d8-ac15-bb8b753de996 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:15.555714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd25ac-e79a-42d8-ac15-bb8b753de996 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:15.557111 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd25ac-e79a-42d8-ac15-bb8b753de996 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:15.564410 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 656/1301] 10.222.0.44 () {56 vars in 1087 bytes} [Thu Jun 23 19:52:15 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 19:52:17.593844 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b88b0-220e-4c71-bd3f-f2620352b651 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:17.594400 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b88b0-220e-4c71-bd3f-f2620352b651 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:17.594400 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b88b0-220e-4c71-bd3f-f2620352b651 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:17.595769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 646/1302] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:52:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:52:17.601013 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed99a22-7635-482b-8609-8f5ebe707389 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:17.607767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed99a22-7635-482b-8609-8f5ebe707389 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:17.607767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed99a22-7635-482b-8609-8f5ebe707389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:17.617533 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-6ed99a22-7635-482b-8609-8f5ebe707389 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:52:17.618713 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6ed99a22-7635-482b-8609-8f5ebe707389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:52:17.682647 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 657/1303] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:52:17 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 19:52:17.707786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b88b0-220e-4c71-bd3f-f2620352b651 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:17.799546 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:52:17.806038 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b88b0-220e-4c71-bd3f-f2620352b651 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:17.839525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b69bb-af41-4f0d-86ff-02a2c6e43aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:17.840334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b69bb-af41-4f0d-86ff-02a2c6e43aac None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:17.840620 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b69bb-af41-4f0d-86ff-02a2c6e43aac None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:17.840893 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b69bb-af41-4f0d-86ff-02a2c6e43aac None admin] PATH_INFO: `/v3/services/placement` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:17.841535 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818b69bb-af41-4f0d-86ff-02a2c6e43aac None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:17.842951 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818b69bb-af41-4f0d-86ff-02a2c6e43aac None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:17.855843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-818b69bb-af41-4f0d-86ff-02a2c6e43aac None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 19:52:17.855843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:52:17.855843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 19:52:17.855843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 19:52:17.855843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 19:52:17.855843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 19:52:17.855843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 19:52:17.855843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:52:17.855843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 19:52:17.855843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:52:17.855843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 19:52:17.855843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:52:17.855843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 19:52:17.855843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 19:52:17.855843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 19:52:17.855843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 19:52:17.855843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 19:52:17.857015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 19:52:17.857015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 19:52:17.857015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 19:52:17.857015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 19:52:17.857015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 19:52:17.857015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 19:52:17.857015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 19:52:17.857015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 19:52:17.857015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 19:52:17.857015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 19:52:17.857015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 19:52:17.857015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 19:52:17.857015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 19:52:17.857015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 19:52:17.857015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 19:52:17.857015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 19:52:17.857915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 19:52:17.857915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 23 19:52:17.857915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 19:52:17.857915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 23 19:52:17.857915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 19:52:17.857915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 19:52:17.857915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 19:52:17.857915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 23 19:52:17.857915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 19:52:17.857915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 23 19:52:17.857915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 19:52:17.857915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 19:52:17.857915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 19:52:17.857915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 23 19:52:17.857915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 647/1304] 10.222.0.44 () {56 vars in 1117 bytes} [Thu Jun 23 19:52:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 19:52:17.862734 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed99a22-7635-482b-8609-8f5ebe707389 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:17.938288 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:52:17.960963 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed99a22-7635-482b-8609-8f5ebe707389 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:17.993646 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1884e-5a5c-4bab-9f4d-c81e175c4575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:17.994297 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1884e-5a5c-4bab-9f4d-c81e175c4575 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:17.994579 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1884e-5a5c-4bab-9f4d-c81e175c4575 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:17.994847 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1884e-5a5c-4bab-9f4d-c81e175c4575 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:17.995540 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe1884e-5a5c-4bab-9f4d-c81e175c4575 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 19:52:17.995843 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe1884e-5a5c-4bab-9f4d-c81e175c4575 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:17.997232 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe1884e-5a5c-4bab-9f4d-c81e175c4575 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:18.002832 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 658/1305] 10.222.0.44 () {56 vars in 1116 bytes} [Thu Jun 23 19:52:17 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 19:52:18.008554 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b69bb-af41-4f0d-86ff-02a2c6e43aac None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:52:18.105143 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b69bb-af41-4f0d-86ff-02a2c6e43aac None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:52:18.138133 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c3099-00ae-4370-881c-6b00c1460c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:52:18.138719 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c3099-00ae-4370-881c-6b00c1460c6f None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:52:18.139035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c3099-00ae-4370-881c-6b00c1460c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:52:18.139306 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c3099-00ae-4370-881c-6b00c1460c6f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:52:18.140029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c3099-00ae-4370-881c-6b00c1460c6f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:52:18.141431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c3099-00ae-4370-881c-6b00c1460c6f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:52:18.217725 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 648/1306] 10.222.0.44 () {60 vars in 1144 bytes} [Thu Jun 23 19:52:18 2022] POST /identity/v3/endpoints => generated 331 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 19:52:27.999822 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=30 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:52:28.213205 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:58:39.164205 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2e2b5-95c2-42a6-abd6-14d49d58d1bd None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:58:39.164822 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2e2b5-95c2-42a6-abd6-14d49d58d1bd None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:58:39.164822 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2e2b5-95c2-42a6-abd6-14d49d58d1bd None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:58:39.171975 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbaf6d-4caf-4405-8950-af7c3840b2a7 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:58:39.172613 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbaf6d-4caf-4405-8950-af7c3840b2a7 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:58:39.172613 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbaf6d-4caf-4405-8950-af7c3840b2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:58:39.174732 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 659/1307] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:58:39 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:58:39.258730 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-78fbaf6d-4caf-4405-8950-af7c3840b2a7 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:58:39.259875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-78fbaf6d-4caf-4405-8950-af7c3840b2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:58:39.401009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 649/1308] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:58:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 19:58:39.436137 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e13b4-0608-47a8-b5e6-e1218ae4ab26 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:58:39.436454 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e13b4-0608-47a8-b5e6-e1218ae4ab26 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:58:39.436743 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e13b4-0608-47a8-b5e6-e1218ae4ab26 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:58:39.438508 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 660/1309] 10.222.0.44 () {54 vars in 964 bytes} [Thu Jun 23 19:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:58:39.446116 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c496de0-398a-45a8-947a-7b1550bcaf36 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:58:39.446595 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c496de0-398a-45a8-947a-7b1550bcaf36 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:58:39.447030 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c496de0-398a-45a8-947a-7b1550bcaf36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:58:39.558312 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5c496de0-398a-45a8-947a-7b1550bcaf36 None None] MFA Rules not processed for user `332fbbefba114da4a01fdd496fff1e31`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:58:39.560067 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5c496de0-398a-45a8-947a-7b1550bcaf36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:58:39.596808 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:58:39.661396 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e13b4-0608-47a8-b5e6-e1218ae4ab26 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:58:39.663121 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3a1e13b4-0608-47a8-b5e6-e1218ae4ab26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:58:39.672255 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 650/1310] 10.222.0.44 () {58 vars in 1061 bytes} [Thu Jun 23 19:58:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 19:58:39.792491 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e13b4-0608-47a8-b5e6-e1218ae4ab26 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:58:39.872990 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc8265-b866-4df0-8f4c-42e2e171cffe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:58:39.873935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc8265-b866-4df0-8f4c-42e2e171cffe service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:58:39.874227 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc8265-b866-4df0-8f4c-42e2e171cffe service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:58:39.874513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc8265-b866-4df0-8f4c-42e2e171cffe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:58:39.876138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c4dc8265-b866-4df0-8f4c-42e2e171cffe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:58:39.934158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:58:39.992806 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc8265-b866-4df0-8f4c-42e2e171cffe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:58:39.994260 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc8265-b866-4df0-8f4c-42e2e171cffe service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:58:40.015431 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 661/1311] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 19:58:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 19:58:42.394626 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0eff1-601f-4473-b472-9d53ea453046 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:58:42.395163 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0eff1-601f-4473-b472-9d53ea453046 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:58:42.395163 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0eff1-601f-4473-b472-9d53ea453046 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:58:42.397761 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 651/1312] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:58:42.403017 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520ebb7-0e43-439f-bbbc-9c206b2f74e5 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:58:42.403556 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520ebb7-0e43-439f-bbbc-9c206b2f74e5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:58:42.403817 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520ebb7-0e43-439f-bbbc-9c206b2f74e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:58:42.481694 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9520ebb7-0e43-439f-bbbc-9c206b2f74e5 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:58:42.482985 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9520ebb7-0e43-439f-bbbc-9c206b2f74e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:58:42.586207 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 662/1313] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:58:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 19:58:42.625282 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0eff1-601f-4473-b472-9d53ea453046 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:58:42.627004 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-91f0eff1-601f-4473-b472-9d53ea453046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:58:42.750506 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0eff1-601f-4473-b472-9d53ea453046 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:58:42.831925 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:58:42.834456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd7d79-418a-43f0-840b-471cbd7992b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:58:42.835110 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd7d79-418a-43f0-840b-471cbd7992b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:58:42.835391 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd7d79-418a-43f0-840b-471cbd7992b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:58:42.835665 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd7d79-418a-43f0-840b-471cbd7992b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:58:42.837397 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d7bd7d79-418a-43f0-840b-471cbd7992b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:58:42.946417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd7d79-418a-43f0-840b-471cbd7992b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:58:42.947898 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd7d79-418a-43f0-840b-471cbd7992b5 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:58:42.968572 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 652/1314] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 19:58:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 19:58:45.702520 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d94c1-37e7-42c9-be25-6ea9b4249ec1 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:58:45.703082 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d94c1-37e7-42c9-be25-6ea9b4249ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:58:45.703082 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d94c1-37e7-42c9-be25-6ea9b4249ec1 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:58:45.704468 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 663/1315] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:58:45.709550 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58b9e6-0e6f-48c6-8dfa-efbfef0f6776 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:58:45.709959 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58b9e6-0e6f-48c6-8dfa-efbfef0f6776 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:58:45.710100 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58b9e6-0e6f-48c6-8dfa-efbfef0f6776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:58:45.773221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6a58b9e6-0e6f-48c6-8dfa-efbfef0f6776 None None] MFA Rules not processed for user `fa6694f71dec4568a4b7724c3570eff4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:58:45.774435 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6a58b9e6-0e6f-48c6-8dfa-efbfef0f6776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:58:45.826761 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:58:45.869545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 653/1316] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:58:45 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 19:58:45.911567 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d94c1-37e7-42c9-be25-6ea9b4249ec1 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:58:45.913137 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-716d94c1-37e7-42c9-be25-6ea9b4249ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:58:45.923489 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:58:46.016546 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d94c1-37e7-42c9-be25-6ea9b4249ec1 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:58:46.088217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2b281-2e11-42db-aba5-5a1f60fb9c7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:58:46.088887 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2b281-2e11-42db-aba5-5a1f60fb9c7d service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:58:46.089171 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2b281-2e11-42db-aba5-5a1f60fb9c7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:58:46.089445 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2b281-2e11-42db-aba5-5a1f60fb9c7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:58:46.091109 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-97e2b281-2e11-42db-aba5-5a1f60fb9c7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:58:46.151906 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:58:46.197207 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2b281-2e11-42db-aba5-5a1f60fb9c7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:58:46.198651 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2b281-2e11-42db-aba5-5a1f60fb9c7d service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:58:46.216426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 664/1317] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 19:58:45 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 19:58:49.217927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ecfff-8f1f-4c5a-82c4-cba56345221c None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:58:49.218453 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ecfff-8f1f-4c5a-82c4-cba56345221c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:58:49.218453 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ecfff-8f1f-4c5a-82c4-cba56345221c None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:58:49.219868 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 654/1318] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:58:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:58:49.224911 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3a3d9-71a8-434b-902f-9364f6e6aa4b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:58:49.225325 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3a3d9-71a8-434b-902f-9364f6e6aa4b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:58:49.225460 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e3a3d9-71a8-434b-902f-9364f6e6aa4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:58:49.289785 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-24e3a3d9-71a8-434b-902f-9364f6e6aa4b None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:58:49.290997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-24e3a3d9-71a8-434b-902f-9364f6e6aa4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:58:49.390947 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 665/1319] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:58:49 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 19:58:49.438261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36ff18-0e9b-483d-87fa-cc294e350c90 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:58:49.438575 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36ff18-0e9b-483d-87fa-cc294e350c90 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:58:49.438853 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36ff18-0e9b-483d-87fa-cc294e350c90 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:58:49.440329 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 655/1320] 10.222.0.44 () {54 vars in 964 bytes} [Thu Jun 23 19:58:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:58:49.446757 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d0db5-4c44-48d1-b975-8554e7855fb0 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:58:49.447047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d0db5-4c44-48d1-b975-8554e7855fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:58:49.447324 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d0db5-4c44-48d1-b975-8554e7855fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:58:49.468477 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:58:49.520339 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-875d0db5-4c44-48d1-b975-8554e7855fb0 None None] MFA Rules not processed for user `332fbbefba114da4a01fdd496fff1e31`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:58:49.521538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-875d0db5-4c44-48d1-b975-8554e7855fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:58:49.596955 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 666/1321] 10.222.0.44 () {58 vars in 1061 bytes} [Thu Jun 23 19:58:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 19:58:49.606339 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36ff18-0e9b-483d-87fa-cc294e350c90 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:58:49.607941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ad36ff18-0e9b-483d-87fa-cc294e350c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:58:49.712192 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36ff18-0e9b-483d-87fa-cc294e350c90 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:58:49.782836 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff38faa-10ee-4652-9798-4c967afdc756 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:58:49.783514 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff38faa-10ee-4652-9798-4c967afdc756 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:58:49.783800 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff38faa-10ee-4652-9798-4c967afdc756 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:58:49.784078 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff38faa-10ee-4652-9798-4c967afdc756 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:58:49.785760 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fff38faa-10ee-4652-9798-4c967afdc756 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:58:49.793206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:58:49.894084 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff38faa-10ee-4652-9798-4c967afdc756 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:58:49.895538 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff38faa-10ee-4652-9798-4c967afdc756 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:58:49.913335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 656/1322] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 19:58:49 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 19:58:53.865934 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73c6ff-8339-4163-8ef1-dc73a04befbd None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:58:53.866541 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73c6ff-8339-4163-8ef1-dc73a04befbd None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:58:53.866541 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73c6ff-8339-4163-8ef1-dc73a04befbd None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:58:53.867858 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 667/1323] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:58:53.873758 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1528f-6c77-46ac-a935-6527109b874d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:58:53.874223 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1528f-6c77-46ac-a935-6527109b874d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:58:53.874331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1528f-6c77-46ac-a935-6527109b874d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:58:53.939610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-32c1528f-6c77-46ac-a935-6527109b874d None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:58:53.940894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-32c1528f-6c77-46ac-a935-6527109b874d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:58:54.000597 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:58:54.043363 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 657/1324] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:58:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 19:58:54.080604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac73c6ff-8339-4163-8ef1-dc73a04befbd None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:58:54.082240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ac73c6ff-8339-4163-8ef1-dc73a04befbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:58:54.192369 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac73c6ff-8339-4163-8ef1-dc73a04befbd None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:58:54.204740 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:58:54.267194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407f280-0793-44b2-8839-cedd2b346a46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:58:54.267846 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407f280-0793-44b2-8839-cedd2b346a46 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:58:54.268130 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407f280-0793-44b2-8839-cedd2b346a46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:58:54.268451 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407f280-0793-44b2-8839-cedd2b346a46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:58:54.270092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2407f280-0793-44b2-8839-cedd2b346a46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:58:54.378996 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2407f280-0793-44b2-8839-cedd2b346a46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:58:54.383337 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2407f280-0793-44b2-8839-cedd2b346a46 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:58:54.399876 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 668/1325] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 19:58:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 19:58:56.509651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30b37a-f0d4-4b1b-937f-9cba13cc53ce None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:58:56.510182 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30b37a-f0d4-4b1b-937f-9cba13cc53ce None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:58:56.510182 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30b37a-f0d4-4b1b-937f-9cba13cc53ce None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:58:56.511502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 658/1326] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:58:56.517420 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd3a03-377b-41c3-ae88-8ad209c67f23 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:58:56.517859 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd3a03-377b-41c3-ae88-8ad209c67f23 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:58:56.518040 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd3a03-377b-41c3-ae88-8ad209c67f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:58:56.563223 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:58:56.581377 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f4fd3a03-377b-41c3-ae88-8ad209c67f23 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:58:56.584821 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f4fd3a03-377b-41c3-ae88-8ad209c67f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:58:56.678256 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 669/1327] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:58:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 19:58:56.713881 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30b37a-f0d4-4b1b-937f-9cba13cc53ce None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:58:56.715410 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2c30b37a-f0d4-4b1b-937f-9cba13cc53ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:58:56.821009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30b37a-f0d4-4b1b-937f-9cba13cc53ce None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:58:56.906196 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ab55a-b74b-4a2e-a21a-1a7a9ddb3257 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:58:56.906850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ab55a-b74b-4a2e-a21a-1a7a9ddb3257 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:58:56.907135 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ab55a-b74b-4a2e-a21a-1a7a9ddb3257 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:58:56.907411 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ab55a-b74b-4a2e-a21a-1a7a9ddb3257 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:58:56.909096 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fc8ab55a-b74b-4a2e-a21a-1a7a9ddb3257 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:58:56.926300 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:58:57.019787 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ab55a-b74b-4a2e-a21a-1a7a9ddb3257 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:58:57.021268 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8ab55a-b74b-4a2e-a21a-1a7a9ddb3257 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:58:57.040191 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 659/1328] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 19:58:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 19:58:59.140681 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d89674-ec2f-4292-baf9-2ddafabaea97 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:58:59.141239 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d89674-ec2f-4292-baf9-2ddafabaea97 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:58:59.141239 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d89674-ec2f-4292-baf9-2ddafabaea97 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:58:59.142596 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 670/1329] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:58:59.147827 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a932643-9782-4565-bc70-6d3accf6e0a2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:58:59.148289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a932643-9782-4565-bc70-6d3accf6e0a2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:58:59.148535 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a932643-9782-4565-bc70-6d3accf6e0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:58:59.214048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-0a932643-9782-4565-bc70-6d3accf6e0a2 None None] MFA Rules not processed for user `fa6694f71dec4568a4b7724c3570eff4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:58:59.215754 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0a932643-9782-4565-bc70-6d3accf6e0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:58:59.282871 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:58:59.318732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 660/1330] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:58:59 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 19:58:59.358118 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d89674-ec2f-4292-baf9-2ddafabaea97 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:58:59.359666 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a4d89674-ec2f-4292-baf9-2ddafabaea97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:58:59.463473 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d89674-ec2f-4292-baf9-2ddafabaea97 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:58:59.484212 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:58:59.539747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4184f89d-007a-48e0-98a4-82c9e1ddccdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:58:59.540426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184f89d-007a-48e0-98a4-82c9e1ddccdf service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:58:59.540710 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184f89d-007a-48e0-98a4-82c9e1ddccdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:58:59.540981 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184f89d-007a-48e0-98a4-82c9e1ddccdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:58:59.542620 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4184f89d-007a-48e0-98a4-82c9e1ddccdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:58:59.650679 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184f89d-007a-48e0-98a4-82c9e1ddccdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:58:59.652138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4184f89d-007a-48e0-98a4-82c9e1ddccdf service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:58:59.670833 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 671/1331] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 19:58:59 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 19:59:02.277770 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943fac10-d3a3-484b-8718-835355fbeb7e None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:02.278269 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943fac10-d3a3-484b-8718-835355fbeb7e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:02.278367 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943fac10-d3a3-484b-8718-835355fbeb7e None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:02.280006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 661/1332] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:59:02.285212 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481a68b-72e3-401c-b165-6d1513d8be1a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:02.285638 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481a68b-72e3-401c-b165-6d1513d8be1a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:02.285764 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481a68b-72e3-401c-b165-6d1513d8be1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:02.348486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:59:02.350982 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-2481a68b-72e3-401c-b165-6d1513d8be1a None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:59:02.352185 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2481a68b-72e3-401c-b165-6d1513d8be1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:02.447406 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 672/1333] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:59:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 19:59:02.483195 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943fac10-d3a3-484b-8718-835355fbeb7e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:59:02.484756 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-943fac10-d3a3-484b-8718-835355fbeb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:02.586867 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943fac10-d3a3-484b-8718-835355fbeb7e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:59:02.661178 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed622ae7-6664-4f89-afe1-729516686cc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:59:02.661774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed622ae7-6664-4f89-afe1-729516686cc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:02.662054 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed622ae7-6664-4f89-afe1-729516686cc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:02.662325 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed622ae7-6664-4f89-afe1-729516686cc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:02.663981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ed622ae7-6664-4f89-afe1-729516686cc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:02.690640 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:59:02.770265 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed622ae7-6664-4f89-afe1-729516686cc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:59:02.771715 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed622ae7-6664-4f89-afe1-729516686cc6 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:59:02.789718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 662/1334] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 19:59:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 19:59:04.881294 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e7f4a-036a-49d9-8683-2590adfa6732 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:04.881836 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e7f4a-036a-49d9-8683-2590adfa6732 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:04.881836 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e7f4a-036a-49d9-8683-2590adfa6732 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:04.883219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 673/1335] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:59:04.888310 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5c878-082f-47e1-afbf-b5039e569c2e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:04.888709 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5c878-082f-47e1-afbf-b5039e569c2e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:04.889410 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5c878-082f-47e1-afbf-b5039e569c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:04.955119 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e4a5c878-082f-47e1-afbf-b5039e569c2e None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:59:04.956361 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e4a5c878-082f-47e1-afbf-b5039e569c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:05.028983 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:59:05.051031 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 663/1336] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:59:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 19:59:05.090920 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e7f4a-036a-49d9-8683-2590adfa6732 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:59:05.093055 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bf2e7f4a-036a-49d9-8683-2590adfa6732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:05.193308 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e7f4a-036a-49d9-8683-2590adfa6732 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:59:05.220958 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:59:05.266258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d3c54-828f-4203-a3bd-cb0b8820f170 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:59:05.266899 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d3c54-828f-4203-a3bd-cb0b8820f170 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:05.267176 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d3c54-828f-4203-a3bd-cb0b8820f170 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:05.267446 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d3c54-828f-4203-a3bd-cb0b8820f170 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:05.269101 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-769d3c54-828f-4203-a3bd-cb0b8820f170 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:05.373645 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d3c54-828f-4203-a3bd-cb0b8820f170 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:59:05.375099 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d3c54-828f-4203-a3bd-cb0b8820f170 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:59:05.393484 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 674/1337] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 19:59:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 19:59:08.275796 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbef606-f465-411f-b459-62aa90c36831 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:08.276332 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbef606-f465-411f-b459-62aa90c36831 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:08.276457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbef606-f465-411f-b459-62aa90c36831 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:08.277751 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 664/1338] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:59:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:59:08.284421 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b08bfc-e1ce-4031-821b-ffc9621bcd4d None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:08.284891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b08bfc-e1ce-4031-821b-ffc9621bcd4d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:08.284988 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b08bfc-e1ce-4031-821b-ffc9621bcd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:08.350927 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-19b08bfc-e1ce-4031-821b-ffc9621bcd4d None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:59:08.352122 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-19b08bfc-e1ce-4031-821b-ffc9621bcd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:08.360660 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:59:08.447049 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 675/1339] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:59:08 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 19:59:08.483841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbef606-f465-411f-b459-62aa90c36831 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:59:08.489022 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4fbef606-f465-411f-b459-62aa90c36831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:08.586568 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbef606-f465-411f-b459-62aa90c36831 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:59:08.658908 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d53e8a-f666-4d00-9cd6-f14eef93a5c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:59:08.659552 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d53e8a-f666-4d00-9cd6-f14eef93a5c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:08.659834 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d53e8a-f666-4d00-9cd6-f14eef93a5c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:08.660108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d53e8a-f666-4d00-9cd6-f14eef93a5c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:08.661783 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f0d53e8a-f666-4d00-9cd6-f14eef93a5c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:08.697677 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:59:08.769263 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d53e8a-f666-4d00-9cd6-f14eef93a5c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:59:08.770631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d53e8a-f666-4d00-9cd6-f14eef93a5c9 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:59:08.788136 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 665/1340] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 19:59:08 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 19:59:13.085258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1005f5d-d816-4104-9ca6-9a1ede9b1b7d None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:13.085709 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1005f5d-d816-4104-9ca6-9a1ede9b1b7d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:13.085889 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1005f5d-d816-4104-9ca6-9a1ede9b1b7d None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:13.087171 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 676/1341] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:59:13.092366 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a0b0f-2582-43f5-bb81-342b91727926 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:13.092776 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a0b0f-2582-43f5-bb81-342b91727926 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:13.092917 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a0b0f-2582-43f5-bb81-342b91727926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:13.181702 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-791a0b0f-2582-43f5-bb81-342b91727926 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:59:13.183141 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-791a0b0f-2582-43f5-bb81-342b91727926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:13.280036 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:59:13.292649 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 666/1342] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:59:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 19:59:13.332332 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1005f5d-d816-4104-9ca6-9a1ede9b1b7d None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:59:13.334448 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c1005f5d-d816-4104-9ca6-9a1ede9b1b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:13.431236 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1005f5d-d816-4104-9ca6-9a1ede9b1b7d None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:59:13.434971 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e750b1-d4a4-495c-958e-c60f54f39fd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:59:13.435644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e750b1-d4a4-495c-958e-c60f54f39fd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:13.435925 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e750b1-d4a4-495c-958e-c60f54f39fd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:13.436199 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e750b1-d4a4-495c-958e-c60f54f39fd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:13.445663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e750b1-d4a4-495c-958e-c60f54f39fd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:59:13.447524 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e750b1-d4a4-495c-958e-c60f54f39fd0 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:59:13.457151 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 677/1343] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 19:59:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 19:59:16.507751 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db9316-76a9-48ad-bb1b-62f702b199ce None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:16.508276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db9316-76a9-48ad-bb1b-62f702b199ce None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:16.508276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db9316-76a9-48ad-bb1b-62f702b199ce None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:16.509633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 667/1344] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:59:16.515195 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98e6dd-3924-4b00-9dc0-9a042c2a23b9 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:16.515624 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98e6dd-3924-4b00-9dc0-9a042c2a23b9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:16.515730 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98e6dd-3924-4b00-9dc0-9a042c2a23b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:16.543702 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-6a98e6dd-3924-4b00-9dc0-9a042c2a23b9 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:59:16.545490 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6a98e6dd-3924-4b00-9dc0-9a042c2a23b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:16.572631 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:59:16.621675 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 678/1345] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:59:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 19:59:16.659500 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db9316-76a9-48ad-bb1b-62f702b199ce None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:59:16.670680 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db9316-76a9-48ad-bb1b-62f702b199ce None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:59:16.674263 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a01f52-bd45-475e-853c-ba5df4f7e3f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:59:16.674895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a01f52-bd45-475e-853c-ba5df4f7e3f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:16.675182 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a01f52-bd45-475e-853c-ba5df4f7e3f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:16.675452 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a01f52-bd45-475e-853c-ba5df4f7e3f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:16.684405 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a01f52-bd45-475e-853c-ba5df4f7e3f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:59:16.685881 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a01f52-bd45-475e-853c-ba5df4f7e3f0 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:59:16.695297 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 668/1346] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 19:59:16 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 19:59:20.934227 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69ef67-036f-45f9-a52b-66d2946f2413 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:20.934794 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69ef67-036f-45f9-a52b-66d2946f2413 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:20.934794 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69ef67-036f-45f9-a52b-66d2946f2413 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:20.936675 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 679/1347] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:59:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:59:20.941769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d3832-8d82-4f18-8912-71a625751909 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:20.942203 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d3832-8d82-4f18-8912-71a625751909 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:20.942422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d3832-8d82-4f18-8912-71a625751909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:20.960975 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a56d3832-8d82-4f18-8912-71a625751909 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:59:20.962201 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a56d3832-8d82-4f18-8912-71a625751909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:20.970813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 669/1348] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:59:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 19:59:21.008729 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69ef67-036f-45f9-a52b-66d2946f2413 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:59:21.010522 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-db69ef67-036f-45f9-a52b-66d2946f2413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:21.032116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69ef67-036f-45f9-a52b-66d2946f2413 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:59:21.035847 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51612c-a00b-4e1e-8b4c-e59eedac7ddf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:59:21.036727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51612c-a00b-4e1e-8b4c-e59eedac7ddf service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:21.037035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51612c-a00b-4e1e-8b4c-e59eedac7ddf service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:21.037374 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51612c-a00b-4e1e-8b4c-e59eedac7ddf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:21.046916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51612c-a00b-4e1e-8b4c-e59eedac7ddf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:59:21.048485 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51612c-a00b-4e1e-8b4c-e59eedac7ddf service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:59:21.058228 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 680/1349] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 19:59:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 19:59:26.093315 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6ae67-7ef6-4532-9a9f-1c89fc216324 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:26.093879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6ae67-7ef6-4532-9a9f-1c89fc216324 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:26.094084 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6ae67-7ef6-4532-9a9f-1c89fc216324 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:26.095752 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 670/1350] 10.222.0.44 () {54 vars in 921 bytes} [Thu Jun 23 19:59:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:59:26.101269 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d7cdd-326a-4eaa-8101-b51db482dc92 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:26.102176 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d7cdd-326a-4eaa-8101-b51db482dc92 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:26.102663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d7cdd-326a-4eaa-8101-b51db482dc92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:26.211513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-388d7cdd-326a-4eaa-8101-b51db482dc92 None None] MFA Rules not processed for user `9715d5deeccd419b97cb0bd39e17982f`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:59:26.213072 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-388d7cdd-326a-4eaa-8101-b51db482dc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:26.346543 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 681/1351] 10.222.0.44 () {58 vars in 1018 bytes} [Thu Jun 23 19:59:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 19:59:30.719284 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d637e40-458a-4951-af82-b2895ff3cd19 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:30.719829 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d637e40-458a-4951-af82-b2895ff3cd19 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:30.719829 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d637e40-458a-4951-af82-b2895ff3cd19 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:30.721425 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 671/1352] 10.222.0.44 () {54 vars in 921 bytes} [Thu Jun 23 19:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:59:30.726733 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3299afc-314e-4217-8485-2b10ea443a10 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:30.727177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3299afc-314e-4217-8485-2b10ea443a10 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:30.727501 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3299afc-314e-4217-8485-2b10ea443a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:30.747698 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a3299afc-314e-4217-8485-2b10ea443a10 None None] MFA Rules not processed for user `9715d5deeccd419b97cb0bd39e17982f`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:59:30.749180 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a3299afc-314e-4217-8485-2b10ea443a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:30.757937 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 682/1353] 10.222.0.44 () {58 vars in 1018 bytes} [Thu Jun 23 19:59:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 19:59:30.955727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=13 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:59:32.410338 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51205e8-517c-4251-915e-653f2a255139 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:32.410953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51205e8-517c-4251-915e-653f2a255139 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:32.410953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51205e8-517c-4251-915e-653f2a255139 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:32.412346 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 672/1354] 10.222.0.44 () {54 vars in 921 bytes} [Thu Jun 23 19:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:59:32.417890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45099b-c5e7-4e02-93f4-73acc6b3514d None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:32.418345 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45099b-c5e7-4e02-93f4-73acc6b3514d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:32.418455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45099b-c5e7-4e02-93f4-73acc6b3514d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:32.437346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ab45099b-c5e7-4e02-93f4-73acc6b3514d None None] MFA Rules not processed for user `9715d5deeccd419b97cb0bd39e17982f`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:59:32.438548 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ab45099b-c5e7-4e02-93f4-73acc6b3514d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:32.447877 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 683/1355] 10.222.0.44 () {58 vars in 1018 bytes} [Thu Jun 23 19:59:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 19:59:36.096582 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf0c78-4f8d-43e3-9efd-481b21dd8449 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:36.097459 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf0c78-4f8d-43e3-9efd-481b21dd8449 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:36.097974 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf0c78-4f8d-43e3-9efd-481b21dd8449 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:36.099976 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 673/1356] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:59:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:59:36.106623 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266d5af-c1f6-4ee2-bde1-298863b04030 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:36.107141 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266d5af-c1f6-4ee2-bde1-298863b04030 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:36.107459 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266d5af-c1f6-4ee2-bde1-298863b04030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:36.152157 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-5266d5af-c1f6-4ee2-bde1-298863b04030 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:59:36.153442 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5266d5af-c1f6-4ee2-bde1-298863b04030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:36.162562 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 684/1357] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:59:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 19:59:36.217489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ea5e1-a79a-46da-a72e-94a384fa8aba None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:36.218125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ea5e1-a79a-46da-a72e-94a384fa8aba None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:36.218593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ea5e1-a79a-46da-a72e-94a384fa8aba None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:36.221082 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 674/1358] 10.222.0.44 () {54 vars in 961 bytes} [Thu Jun 23 19:59:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:59:36.242376 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9001fab8-3555-4903-9bca-3174477969a9 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:36.242666 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9001fab8-3555-4903-9bca-3174477969a9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:36.243150 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9001fab8-3555-4903-9bca-3174477969a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:36.295750 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:59:36.304328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9001fab8-3555-4903-9bca-3174477969a9 None None] MFA Rules not processed for user `a63d1d902c024c97a38f7d92f73814d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:59:36.305706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9001fab8-3555-4903-9bca-3174477969a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:36.407457 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 685/1359] 10.222.0.44 () {58 vars in 1058 bytes} [Thu Jun 23 19:59:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 19:59:36.415560 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ea5e1-a79a-46da-a72e-94a384fa8aba None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:59:36.430699 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ea5e1-a79a-46da-a72e-94a384fa8aba None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:59:36.437260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f2519b-17f8-4187-9013-5dc42c426a4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:59:36.438120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f2519b-17f8-4187-9013-5dc42c426a4d service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:36.438412 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f2519b-17f8-4187-9013-5dc42c426a4d service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:36.438683 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f2519b-17f8-4187-9013-5dc42c426a4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:36.448753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f2519b-17f8-4187-9013-5dc42c426a4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:59:36.450636 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f2519b-17f8-4187-9013-5dc42c426a4d service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:59:36.460598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 675/1360] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 19:59:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 19:59:38.776958 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbd710-bb21-48d3-88bb-8e2d27515b6c None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:38.777484 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbd710-bb21-48d3-88bb-8e2d27515b6c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:38.777598 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbd710-bb21-48d3-88bb-8e2d27515b6c None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:38.779119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 686/1361] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:59:38.784446 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad02ab-4c65-4454-a875-d4edf0db3e85 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:38.784864 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad02ab-4c65-4454-a875-d4edf0db3e85 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:38.785003 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad02ab-4c65-4454-a875-d4edf0db3e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:38.803763 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4fad02ab-4c65-4454-a875-d4edf0db3e85 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:59:38.805065 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4fad02ab-4c65-4454-a875-d4edf0db3e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:38.813150 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 676/1362] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:59:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 19:59:38.859963 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fbd710-bb21-48d3-88bb-8e2d27515b6c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:59:38.875657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fbd710-bb21-48d3-88bb-8e2d27515b6c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:59:38.885616 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b690bd4-04fa-4f41-8855-7f098b6e19e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:59:38.886356 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b690bd4-04fa-4f41-8855-7f098b6e19e9 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:38.886738 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b690bd4-04fa-4f41-8855-7f098b6e19e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:38.887079 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b690bd4-04fa-4f41-8855-7f098b6e19e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:38.900537 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b690bd4-04fa-4f41-8855-7f098b6e19e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:59:38.902279 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b690bd4-04fa-4f41-8855-7f098b6e19e9 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:59:38.913413 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 687/1363] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 19:59:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 19:59:41.499994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff97d3-40cf-44fd-bf35-f9deab9d00c0 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:41.500535 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff97d3-40cf-44fd-bf35-f9deab9d00c0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:41.500535 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff97d3-40cf-44fd-bf35-f9deab9d00c0 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:41.501906 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 677/1364] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:59:41.508598 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfa883-ddc7-41a5-a7df-2805a9dfbc4a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:41.509032 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfa883-ddc7-41a5-a7df-2805a9dfbc4a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:41.509218 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfa883-ddc7-41a5-a7df-2805a9dfbc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:41.528203 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-66cfa883-ddc7-41a5-a7df-2805a9dfbc4a None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:59:41.529558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-66cfa883-ddc7-41a5-a7df-2805a9dfbc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:41.537829 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 688/1365] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 19:59:41.598196 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f2fb7-ab62-45a1-99cc-1685f73d7c72 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:41.598562 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f2fb7-ab62-45a1-99cc-1685f73d7c72 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:41.598842 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f2fb7-ab62-45a1-99cc-1685f73d7c72 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:41.602906 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 678/1366] 10.222.0.44 () {54 vars in 961 bytes} [Thu Jun 23 19:59:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:59:41.610395 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872db0f2-545c-4521-87d7-2b4d86bf5896 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:41.610689 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872db0f2-545c-4521-87d7-2b4d86bf5896 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:41.610966 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872db0f2-545c-4521-87d7-2b4d86bf5896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:41.629511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-872db0f2-545c-4521-87d7-2b4d86bf5896 None None] MFA Rules not processed for user `a63d1d902c024c97a38f7d92f73814d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:59:41.630730 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-872db0f2-545c-4521-87d7-2b4d86bf5896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:41.640345 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 689/1367] 10.222.0.44 () {58 vars in 1058 bytes} [Thu Jun 23 19:59:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 19:59:41.648275 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f2fb7-ab62-45a1-99cc-1685f73d7c72 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:59:41.662461 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f2fb7-ab62-45a1-99cc-1685f73d7c72 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:59:41.667811 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b4b42f-2539-452b-9cba-146aa1c1b808 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:59:41.668488 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4b42f-2539-452b-9cba-146aa1c1b808 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:41.668773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4b42f-2539-452b-9cba-146aa1c1b808 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:41.669048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4b42f-2539-452b-9cba-146aa1c1b808 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:41.678249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b4b42f-2539-452b-9cba-146aa1c1b808 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:59:41.679836 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b4b42f-2539-452b-9cba-146aa1c1b808 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:59:41.690355 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 679/1368] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 19:59:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 19:59:43.995000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2d711-1aa6-44e3-854b-12202926b091 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:43.996061 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2d711-1aa6-44e3-854b-12202926b091 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:43.996606 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2d711-1aa6-44e3-854b-12202926b091 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:43.998242 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 690/1369] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:59:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:59:44.003871 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fdfa6-c566-409c-addb-f54d29a7e15b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:44.004405 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fdfa6-c566-409c-addb-f54d29a7e15b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:44.004510 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200fdfa6-c566-409c-addb-f54d29a7e15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:44.024544 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-200fdfa6-c566-409c-addb-f54d29a7e15b None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:59:44.026195 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-200fdfa6-c566-409c-addb-f54d29a7e15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:44.035976 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 680/1370] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:59:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 19:59:44.080152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2d711-1aa6-44e3-854b-12202926b091 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:59:44.095403 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2d711-1aa6-44e3-854b-12202926b091 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:59:44.101198 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d6533-64ad-4146-ad03-b8524f923546 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:59:44.101998 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d6533-64ad-4146-ad03-b8524f923546 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:44.102374 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d6533-64ad-4146-ad03-b8524f923546 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:44.102745 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d6533-64ad-4146-ad03-b8524f923546 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:44.111880 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d6533-64ad-4146-ad03-b8524f923546 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:59:44.113773 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d6533-64ad-4146-ad03-b8524f923546 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:59:44.124029 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 691/1371] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 19:59:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 19:59:44.578422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64c31c-1f4a-4c3f-9b3a-6088808b6774 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:44.578709 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64c31c-1f4a-4c3f-9b3a-6088808b6774 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:44.579166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64c31c-1f4a-4c3f-9b3a-6088808b6774 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:44.580579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 681/1372] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:59:44.586042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d71950-2d77-4070-b95c-1f8f17c35918 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:44.586332 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d71950-2d77-4070-b95c-1f8f17c35918 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:44.586612 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d71950-2d77-4070-b95c-1f8f17c35918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:44.604462 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-84d71950-2d77-4070-b95c-1f8f17c35918 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:59:44.605647 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-84d71950-2d77-4070-b95c-1f8f17c35918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:44.614237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 692/1373] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:59:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 19:59:44.931544 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32abd06-35e6-4051-be6b-0a07c3d1f455 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:44.931845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32abd06-35e6-4051-be6b-0a07c3d1f455 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:44.932087 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32abd06-35e6-4051-be6b-0a07c3d1f455 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:44.933754 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 682/1374] 10.222.0.44 () {54 vars in 963 bytes} [Thu Jun 23 19:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:59:44.940141 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf93ad3-7f9e-4d24-abc9-34e00ae723fb None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:44.941483 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf93ad3-7f9e-4d24-abc9-34e00ae723fb None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:44.941483 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf93ad3-7f9e-4d24-abc9-34e00ae723fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:45.027660 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-eaf93ad3-7f9e-4d24-abc9-34e00ae723fb None None] MFA Rules not processed for user `21ae3b8971b44d928e9f6c0539858636`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:59:45.030661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-eaf93ad3-7f9e-4d24-abc9-34e00ae723fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:45.106501 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 693/1375] 10.222.0.44 () {58 vars in 1060 bytes} [Thu Jun 23 19:59:44 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 19:59:45.115418 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32abd06-35e6-4051-be6b-0a07c3d1f455 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:59:45.127949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32abd06-35e6-4051-be6b-0a07c3d1f455 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:59:45.131709 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0a128-927b-4f10-a09b-c67853e421b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:59:45.132378 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0a128-927b-4f10-a09b-c67853e421b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:45.132663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0a128-927b-4f10-a09b-c67853e421b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:45.133232 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0a128-927b-4f10-a09b-c67853e421b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:45.143134 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec0a128-927b-4f10-a09b-c67853e421b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:59:45.145516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec0a128-927b-4f10-a09b-c67853e421b5 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:59:45.156650 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 683/1376] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 19:59:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 19:59:46.330986 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af174f5-7d3d-4ef7-a747-d49bde50bffc None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:46.331519 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af174f5-7d3d-4ef7-a747-d49bde50bffc None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:46.331665 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af174f5-7d3d-4ef7-a747-d49bde50bffc None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:46.333068 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 694/1377] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:59:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:59:46.343880 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4636e-9f4d-4095-b148-d9c90d3b46a2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:46.344434 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4636e-9f4d-4095-b148-d9c90d3b46a2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:46.344542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc4636e-9f4d-4095-b148-d9c90d3b46a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:46.362753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-3dc4636e-9f4d-4095-b148-d9c90d3b46a2 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:59:46.366126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3dc4636e-9f4d-4095-b148-d9c90d3b46a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:46.373032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 684/1378] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:59:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 19:59:46.417071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af174f5-7d3d-4ef7-a747-d49bde50bffc None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:59:46.432118 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af174f5-7d3d-4ef7-a747-d49bde50bffc None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:59:46.437485 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ff988-fd03-4746-8c0f-f49acb6558a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:59:46.438109 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ff988-fd03-4746-8c0f-f49acb6558a8 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:46.438388 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ff988-fd03-4746-8c0f-f49acb6558a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:46.438660 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ff988-fd03-4746-8c0f-f49acb6558a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:46.447397 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ff988-fd03-4746-8c0f-f49acb6558a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:59:46.449067 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ff988-fd03-4746-8c0f-f49acb6558a8 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:59:46.453120 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 19:59:46.459738 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 695/1379] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 19:59:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 19:59:48.921130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af633e-b396-4188-b9be-dbf2ff5e24a4 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:48.922236 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af633e-b396-4188-b9be-dbf2ff5e24a4 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:48.922792 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af633e-b396-4188-b9be-dbf2ff5e24a4 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:48.924965 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 685/1380] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:59:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:59:48.930453 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef8bc5-3f10-4e3b-9ecf-f9e23c0fa69d None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:48.930977 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef8bc5-3f10-4e3b-9ecf-f9e23c0fa69d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:48.931062 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef8bc5-3f10-4e3b-9ecf-f9e23c0fa69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:48.956553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-72ef8bc5-3f10-4e3b-9ecf-f9e23c0fa69d None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:59:48.958021 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-72ef8bc5-3f10-4e3b-9ecf-f9e23c0fa69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:48.972400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 696/1381] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:59:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 19:59:49.018108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af633e-b396-4188-b9be-dbf2ff5e24a4 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:59:49.039809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8af633e-b396-4188-b9be-dbf2ff5e24a4 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:59:49.050276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad5499-f4b1-45ed-bf68-19f0db7e203c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:59:49.051157 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad5499-f4b1-45ed-bf68-19f0db7e203c service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:49.051767 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad5499-f4b1-45ed-bf68-19f0db7e203c service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:49.052268 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad5499-f4b1-45ed-bf68-19f0db7e203c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:49.062714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ad5499-f4b1-45ed-bf68-19f0db7e203c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:59:49.064340 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ad5499-f4b1-45ed-bf68-19f0db7e203c service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:59:49.074670 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 686/1382] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 19:59:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 19:59:51.375712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197294c-2468-4899-ae82-28dbc109cefe None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:51.376227 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197294c-2468-4899-ae82-28dbc109cefe None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:51.376328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197294c-2468-4899-ae82-28dbc109cefe None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:51.377652 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 697/1383] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:59:51.383008 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfddd55-2f29-44b0-aa7b-8370de0dcaf0 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:51.383428 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfddd55-2f29-44b0-aa7b-8370de0dcaf0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:51.383556 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfddd55-2f29-44b0-aa7b-8370de0dcaf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:51.414758 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ccfddd55-2f29-44b0-aa7b-8370de0dcaf0 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:59:51.414758 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ccfddd55-2f29-44b0-aa7b-8370de0dcaf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:51.414758 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 687/1384] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:59:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 19:59:51.455184 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197294c-2468-4899-ae82-28dbc109cefe None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:59:51.470067 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197294c-2468-4899-ae82-28dbc109cefe None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:59:51.475516 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5267f1-ae41-4abb-8889-eefa95951104 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:59:51.476146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5267f1-ae41-4abb-8889-eefa95951104 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:51.476461 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5267f1-ae41-4abb-8889-eefa95951104 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:51.476738 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5267f1-ae41-4abb-8889-eefa95951104 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:51.485119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5267f1-ae41-4abb-8889-eefa95951104 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:59:51.486718 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5267f1-ae41-4abb-8889-eefa95951104 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:59:51.499414 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 698/1385] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 19:59:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 19:59:53.622144 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffdbd64-3624-44a7-8809-7bcc7537b652 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:53.622783 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffdbd64-3624-44a7-8809-7bcc7537b652 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:53.622893 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffdbd64-3624-44a7-8809-7bcc7537b652 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:53.624900 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 688/1386] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:59:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:59:53.630229 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921e893-2f72-45e0-aeac-cb6c6ff4d26f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:53.630651 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921e893-2f72-45e0-aeac-cb6c6ff4d26f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:53.630791 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921e893-2f72-45e0-aeac-cb6c6ff4d26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:53.649643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b921e893-2f72-45e0-aeac-cb6c6ff4d26f None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:59:53.650875 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b921e893-2f72-45e0-aeac-cb6c6ff4d26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:53.661213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 699/1387] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:59:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 19:59:53.712977 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffdbd64-3624-44a7-8809-7bcc7537b652 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:59:53.728965 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffdbd64-3624-44a7-8809-7bcc7537b652 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:59:53.735126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab5563-7f15-4b90-a901-e1017df381dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:59:53.736131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab5563-7f15-4b90-a901-e1017df381dd service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:53.736454 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab5563-7f15-4b90-a901-e1017df381dd service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:53.736735 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab5563-7f15-4b90-a901-e1017df381dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:53.746581 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab5563-7f15-4b90-a901-e1017df381dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:59:53.748854 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab5563-7f15-4b90-a901-e1017df381dd service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:59:53.761426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 689/1388] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 19:59:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 19:59:55.985379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1cee1-157c-4c74-8afc-252c1c9aa933 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:55.985379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1cee1-157c-4c74-8afc-252c1c9aa933 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:55.986223 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1cee1-157c-4c74-8afc-252c1c9aa933 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:55.986854 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 700/1389] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:59:55.992651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fc7b7-332b-4df7-baec-d3c9c8f3efac None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:55.993150 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fc7b7-332b-4df7-baec-d3c9c8f3efac None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:55.993484 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fc7b7-332b-4df7-baec-d3c9c8f3efac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:56.016644 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d02fc7b7-332b-4df7-baec-d3c9c8f3efac None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:59:56.018424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d02fc7b7-332b-4df7-baec-d3c9c8f3efac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:56.027700 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 690/1390] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:59:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 19:59:56.081224 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1cee1-157c-4c74-8afc-252c1c9aa933 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:59:56.096990 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1cee1-157c-4c74-8afc-252c1c9aa933 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:59:56.102626 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6dd079-ad90-41d9-a80c-4214fd83b7b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:59:56.103258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6dd079-ad90-41d9-a80c-4214fd83b7b1 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:56.103549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6dd079-ad90-41d9-a80c-4214fd83b7b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:56.103834 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6dd079-ad90-41d9-a80c-4214fd83b7b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:56.112871 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6dd079-ad90-41d9-a80c-4214fd83b7b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:59:56.114936 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6dd079-ad90-41d9-a80c-4214fd83b7b1 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:59:56.127421 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 701/1391] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 19:59:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 19:59:58.359903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faddc56-7a3b-42a0-94eb-caf36bbffecf None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:58.360487 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faddc56-7a3b-42a0-94eb-caf36bbffecf None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:58.360721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faddc56-7a3b-42a0-94eb-caf36bbffecf None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:58.362524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 691/1392] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 19:59:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 19:59:58.368774 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dafef9-09d1-4adc-8fe4-62878682fb4e None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:58.369209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dafef9-09d1-4adc-8fe4-62878682fb4e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:58.369403 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dafef9-09d1-4adc-8fe4-62878682fb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:58.395085 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-07dafef9-09d1-4adc-8fe4-62878682fb4e None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 19:59:58.396578 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-07dafef9-09d1-4adc-8fe4-62878682fb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 19:59:58.410012 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 702/1393] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 19:59:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 19:59:58.458340 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faddc56-7a3b-42a0-94eb-caf36bbffecf None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 19:59:58.474899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faddc56-7a3b-42a0-94eb-caf36bbffecf None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 19:59:58.482533 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dae2de-fcc4-4b9d-8a25-677181b32908 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 19:59:58.483369 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dae2de-fcc4-4b9d-8a25-677181b32908 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 19:59:58.483735 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dae2de-fcc4-4b9d-8a25-677181b32908 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 19:59:58.484094 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dae2de-fcc4-4b9d-8a25-677181b32908 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 19:59:58.497898 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dae2de-fcc4-4b9d-8a25-677181b32908 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 19:59:58.500225 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dae2de-fcc4-4b9d-8a25-677181b32908 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 19:59:58.514052 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 692/1394] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 19:59:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:00:00.709445 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc981feb-d7e1-4827-b2c3-735084517eb8 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:00.710016 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc981feb-d7e1-4827-b2c3-735084517eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:00.710016 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc981feb-d7e1-4827-b2c3-735084517eb8 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:00.711333 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 703/1395] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 20:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 20:00:00.716998 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8443f-bd3f-499b-add2-386bd8f0b790 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:00.717464 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8443f-bd3f-499b-add2-386bd8f0b790 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:00.717561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8443f-bd3f-499b-add2-386bd8f0b790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:00.735700 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ebc8443f-bd3f-499b-add2-386bd8f0b790 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:00:00.736928 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ebc8443f-bd3f-499b-add2-386bd8f0b790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:00:00.744977 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 693/1396] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 20:00:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:00:00.794992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc981feb-d7e1-4827-b2c3-735084517eb8 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:00:00.809223 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc981feb-d7e1-4827-b2c3-735084517eb8 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:00:00.814468 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e26ede9-ab81-4a6a-8f3b-f97afd663910 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:00:00.815092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26ede9-ab81-4a6a-8f3b-f97afd663910 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:00.815368 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26ede9-ab81-4a6a-8f3b-f97afd663910 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:00.815635 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26ede9-ab81-4a6a-8f3b-f97afd663910 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:00.824015 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e26ede9-ab81-4a6a-8f3b-f97afd663910 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:00:00.826177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e26ede9-ab81-4a6a-8f3b-f97afd663910 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:00:00.835571 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 704/1397] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:00:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:00:00.901359 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2376815-2cab-445f-afde-acef56fab708 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:00.901646 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2376815-2cab-445f-afde-acef56fab708 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:00.901919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2376815-2cab-445f-afde-acef56fab708 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:00.903867 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 694/1398] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 20:00:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:00:00.909121 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffcb7c-ef88-4cfa-b3c6-0d8f697a3df9 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:00.909412 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffcb7c-ef88-4cfa-b3c6-0d8f697a3df9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:00.909687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ffcb7c-ef88-4cfa-b3c6-0d8f697a3df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:00.927223 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-32ffcb7c-ef88-4cfa-b3c6-0d8f697a3df9 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:00:00.928450 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-32ffcb7c-ef88-4cfa-b3c6-0d8f697a3df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:00:00.936395 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 705/1399] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 20:00:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:00:00.993161 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea37e2e-e38a-40d5-80f5-96aa741d68e9 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:00.993443 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea37e2e-e38a-40d5-80f5-96aa741d68e9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:00.993713 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea37e2e-e38a-40d5-80f5-96aa741d68e9 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:00.995125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 695/1400] 10.222.0.44 () {54 vars in 962 bytes} [Thu Jun 23 20:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:00:01.000835 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c3726-8bbe-423c-924c-430a6772bf9e None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:01.001128 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c3726-8bbe-423c-924c-430a6772bf9e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:01.001405 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c3726-8bbe-423c-924c-430a6772bf9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:01.051970 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-429c3726-8bbe-423c-924c-430a6772bf9e None None] MFA Rules not processed for user `1dd72f952df34ad8853fe84673a41f45`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:00:01.053282 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-429c3726-8bbe-423c-924c-430a6772bf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:00:01.125852 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 706/1401] 10.222.0.44 () {58 vars in 1059 bytes} [Thu Jun 23 20:00:00 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:00:01.133785 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea37e2e-e38a-40d5-80f5-96aa741d68e9 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:00:01.139476 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=55 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:00:01.145719 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea37e2e-e38a-40d5-80f5-96aa741d68e9 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:00:01.149470 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0684e-ed52-468d-92bb-4a76440d4ab6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd72f952df34ad8853fe84673a41f45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:00:01.150716 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0684e-ed52-468d-92bb-4a76440d4ab6 service glance] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:01.150716 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0684e-ed52-468d-92bb-4a76440d4ab6 service glance] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:01.150900 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0684e-ed52-468d-92bb-4a76440d4ab6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:01.161201 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0684e-ed52-468d-92bb-4a76440d4ab6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:00:01.163003 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0684e-ed52-468d-92bb-4a76440d4ab6 service glance] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:00:01.174852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 696/1402] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:00:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:00:02.151038 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3deb1-ffe2-42d5-bbee-858fed9edcaa None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:02.151492 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3deb1-ffe2-42d5-bbee-858fed9edcaa None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:02.151605 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3deb1-ffe2-42d5-bbee-858fed9edcaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:02.206914 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-70b3deb1-ffe2-42d5-bbee-858fed9edcaa None None] MFA Rules not processed for user `1d37f369749143ce813c693adc5f2ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:00:02.208346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-70b3deb1-ffe2-42d5-bbee-858fed9edcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:00:02.254791 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:00:02.277432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 707/1403] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jun 23 20:00:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:00:02.319173 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f374f7-0ca9-48d1-a298-d39ddff3a783 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:02.319688 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f374f7-0ca9-48d1-a298-d39ddff3a783 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:02.319774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f374f7-0ca9-48d1-a298-d39ddff3a783 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:02.321464 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 697/1404] 10.222.0.44 () {54 vars in 942 bytes} [Thu Jun 23 20:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:00:02.328472 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd728871-1aed-4945-888d-ee7304a4c361 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:02.328763 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd728871-1aed-4945-888d-ee7304a4c361 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:02.329040 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd728871-1aed-4945-888d-ee7304a4c361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:02.378266 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-dd728871-1aed-4945-888d-ee7304a4c361 None None] MFA Rules not processed for user `96db03079816461bb322633f64b9fb9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:00:02.379485 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-dd728871-1aed-4945-888d-ee7304a4c361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:00:02.462868 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 708/1405] 10.222.0.44 () {58 vars in 1039 bytes} [Thu Jun 23 20:00:02 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:00:02.473230 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f374f7-0ca9-48d1-a298-d39ddff3a783 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:00:02.486582 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f374f7-0ca9-48d1-a298-d39ddff3a783 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:00:02.491510 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b88ba-b2ee-46ee-bbee-ab7fe6cbd71b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96db03079816461bb322633f64b9fb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:00:02.492165 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b88ba-b2ee-46ee-bbee-ab7fe6cbd71b service swift] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:02.492481 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b88ba-b2ee-46ee-bbee-ab7fe6cbd71b service swift] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:02.492754 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b88ba-b2ee-46ee-bbee-ab7fe6cbd71b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:02.500690 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b88ba-b2ee-46ee-bbee-ab7fe6cbd71b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:00:02.502280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b88ba-b2ee-46ee-bbee-ab7fe6cbd71b service swift] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:00:02.510542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 698/1406] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jun 23 20:00:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 20:00:03.124809 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425976f-e88f-4027-8c78-fe3366f03ae7 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:03.125098 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425976f-e88f-4027-8c78-fe3366f03ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:03.125375 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425976f-e88f-4027-8c78-fe3366f03ae7 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:03.130020 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 709/1407] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 20:00:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:00:03.132212 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3c000-53a9-4a70-aa82-a5d2885c2c28 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:03.132532 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3c000-53a9-4a70-aa82-a5d2885c2c28 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:03.132810 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3c000-53a9-4a70-aa82-a5d2885c2c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:03.150880 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5ba3c000-53a9-4a70-aa82-a5d2885c2c28 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:00:03.152097 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5ba3c000-53a9-4a70-aa82-a5d2885c2c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:00:03.164347 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 699/1408] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 20:00:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:00:03.210639 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425976f-e88f-4027-8c78-fe3366f03ae7 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:00:03.228350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425976f-e88f-4027-8c78-fe3366f03ae7 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:00:03.243893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f08807-86ea-4512-947a-9059d9c313e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:00:03.244601 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f08807-86ea-4512-947a-9059d9c313e4 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:03.244913 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f08807-86ea-4512-947a-9059d9c313e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:03.245453 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f08807-86ea-4512-947a-9059d9c313e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:03.262043 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f08807-86ea-4512-947a-9059d9c313e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:00:03.264515 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f08807-86ea-4512-947a-9059d9c313e4 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:00:03.276246 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 710/1409] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:00:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:00:13.149833 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=14 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:00:13.271039 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=59 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:00:26.019662 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb758d8e-cab5-43cc-b2db-a3b5e0bc135e None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:26.020243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb758d8e-cab5-43cc-b2db-a3b5e0bc135e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:26.020243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb758d8e-cab5-43cc-b2db-a3b5e0bc135e None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:26.021681 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 700/1410] 10.222.0.44 () {54 vars in 918 bytes} [Thu Jun 23 20:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:00:26.026776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d438053-369e-42ef-a863-492fd98c3811 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:26.027262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d438053-369e-42ef-a863-492fd98c3811 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:26.027364 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d438053-369e-42ef-a863-492fd98c3811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:26.044837 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-6d438053-369e-42ef-a863-492fd98c3811 None None] MFA Rules not processed for user `9715d5deeccd419b97cb0bd39e17982f`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:00:26.046021 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6d438053-369e-42ef-a863-492fd98c3811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:00:26.054250 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 711/1411] 10.222.0.44 () {58 vars in 1015 bytes} [Thu Jun 23 20:00:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:00:26.068547 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f590e-bdce-4b40-837d-d1fc69df71aa None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:26.068835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f590e-bdce-4b40-837d-d1fc69df71aa None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:26.069109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f590e-bdce-4b40-837d-d1fc69df71aa None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:26.070473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 701/1412] 10.222.0.44 () {54 vars in 918 bytes} [Thu Jun 23 20:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:00:26.075621 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805fd86-bd89-4a8d-8932-ecdd7e78e46b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:26.075905 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805fd86-bd89-4a8d-8932-ecdd7e78e46b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:26.076172 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805fd86-bd89-4a8d-8932-ecdd7e78e46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:26.094324 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-6805fd86-bd89-4a8d-8932-ecdd7e78e46b None None] MFA Rules not processed for user `a63d1d902c024c97a38f7d92f73814d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:00:26.095486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6805fd86-bd89-4a8d-8932-ecdd7e78e46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:00:26.104238 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 712/1413] 10.222.0.44 () {58 vars in 1015 bytes} [Thu Jun 23 20:00:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:00:28.604963 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e463a9-d94b-48a3-90bd-db535fe86e28 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:28.605515 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e463a9-d94b-48a3-90bd-db535fe86e28 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:28.605621 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e463a9-d94b-48a3-90bd-db535fe86e28 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:28.607022 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 702/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 20:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:00:28.613775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127edd5-37a2-4d26-ae86-6d2615f99c72 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:28.614221 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127edd5-37a2-4d26-ae86-6d2615f99c72 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:28.614320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127edd5-37a2-4d26-ae86-6d2615f99c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:28.631652 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-5127edd5-37a2-4d26-ae86-6d2615f99c72 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:00:28.632878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5127edd5-37a2-4d26-ae86-6d2615f99c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:00:28.641052 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 713/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 20:00:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:00:28.684110 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e463a9-d94b-48a3-90bd-db535fe86e28 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:00:28.698183 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e463a9-d94b-48a3-90bd-db535fe86e28 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:00:28.704070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298ef322-eb07-43e0-8240-ca1cc8fae8cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:00:28.704730 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ef322-eb07-43e0-8240-ca1cc8fae8cf service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:28.705013 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ef322-eb07-43e0-8240-ca1cc8fae8cf service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:28.705287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ef322-eb07-43e0-8240-ca1cc8fae8cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:28.713376 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298ef322-eb07-43e0-8240-ca1cc8fae8cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:00:28.715485 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298ef322-eb07-43e0-8240-ca1cc8fae8cf service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:00:28.725013 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 703/1416] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:00:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 20:00:29.619371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331553f-5eb0-4652-a874-b33e64fef2d1 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:29.619945 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331553f-5eb0-4652-a874-b33e64fef2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:29.619945 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331553f-5eb0-4652-a874-b33e64fef2d1 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:29.621435 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 714/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 20:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:00:29.626599 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14b287-c0cf-4579-8aab-afd44f79ed3a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:29.627026 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14b287-c0cf-4579-8aab-afd44f79ed3a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:29.627135 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14b287-c0cf-4579-8aab-afd44f79ed3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:29.644703 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6b14b287-c0cf-4579-8aab-afd44f79ed3a None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:00:29.645905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6b14b287-c0cf-4579-8aab-afd44f79ed3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:00:29.654085 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 704/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 20:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:00:29.808355 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331553f-5eb0-4652-a874-b33e64fef2d1 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:00:29.822709 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331553f-5eb0-4652-a874-b33e64fef2d1 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:00:29.828047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428eca9-a1db-435e-99be-04e040d6c9b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:00:29.828703 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428eca9-a1db-435e-99be-04e040d6c9b2 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:29.828990 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428eca9-a1db-435e-99be-04e040d6c9b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:29.829267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428eca9-a1db-435e-99be-04e040d6c9b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:29.837359 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428eca9-a1db-435e-99be-04e040d6c9b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:00:29.838989 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428eca9-a1db-435e-99be-04e040d6c9b2 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:00:29.848874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 715/1419] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:00:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:00:30.735880 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca912e-63c6-452b-a018-7a653ba35592 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:30.736482 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca912e-63c6-452b-a018-7a653ba35592 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:30.736482 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca912e-63c6-452b-a018-7a653ba35592 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:30.737872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 705/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 20:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:00:30.742760 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0959063-6218-4ff0-94b6-af8b51900e15 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:30.743211 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0959063-6218-4ff0-94b6-af8b51900e15 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:30.743316 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0959063-6218-4ff0-94b6-af8b51900e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:30.760617 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e0959063-6218-4ff0-94b6-af8b51900e15 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:00:30.762357 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e0959063-6218-4ff0-94b6-af8b51900e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:00:30.770092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 716/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 20:00:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:00:30.920923 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca912e-63c6-452b-a018-7a653ba35592 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:00:30.935539 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca912e-63c6-452b-a018-7a653ba35592 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:00:30.940862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cccd9a0-cf24-4953-b020-8847302113a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:00:30.941491 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cccd9a0-cf24-4953-b020-8847302113a8 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:30.941770 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cccd9a0-cf24-4953-b020-8847302113a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:30.942051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cccd9a0-cf24-4953-b020-8847302113a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:30.950215 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cccd9a0-cf24-4953-b020-8847302113a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:00:30.951787 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cccd9a0-cf24-4953-b020-8847302113a8 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:00:30.961011 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 706/1422] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:00:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:00:31.841820 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c95ff-0c9c-4c3d-93fe-e5545716bdc4 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:31.842380 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c95ff-0c9c-4c3d-93fe-e5545716bdc4 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:31.842380 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c95ff-0c9c-4c3d-93fe-e5545716bdc4 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:31.843708 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 717/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 20:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:00:31.848630 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe61525-8bb5-40e6-842c-1f75cca891cb None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:31.849029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe61525-8bb5-40e6-842c-1f75cca891cb None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:31.849187 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe61525-8bb5-40e6-842c-1f75cca891cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:31.867142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9fe61525-8bb5-40e6-842c-1f75cca891cb None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:00:31.868365 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9fe61525-8bb5-40e6-842c-1f75cca891cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:00:31.876043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 707/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 20:00:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:00:32.026564 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c95ff-0c9c-4c3d-93fe-e5545716bdc4 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:00:32.041225 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c95ff-0c9c-4c3d-93fe-e5545716bdc4 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:00:32.047044 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c7d4a-4abd-414c-9d4a-202a7df3e075 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:00:32.047664 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c7d4a-4abd-414c-9d4a-202a7df3e075 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:32.047944 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c7d4a-4abd-414c-9d4a-202a7df3e075 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:32.048213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c7d4a-4abd-414c-9d4a-202a7df3e075 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:32.056198 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c7d4a-4abd-414c-9d4a-202a7df3e075 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:00:32.057833 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c7d4a-4abd-414c-9d4a-202a7df3e075 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:00:32.067388 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 718/1425] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:00:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:00:32.921733 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c926c6-7ae4-44a1-8fd6-185358156380 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:32.922324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c926c6-7ae4-44a1-8fd6-185358156380 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:32.922429 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c926c6-7ae4-44a1-8fd6-185358156380 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:32.923786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 708/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 20:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:00:32.928747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcc248-bc07-4ab7-aac6-a952f925386c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:32.929185 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcc248-bc07-4ab7-aac6-a952f925386c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:32.929300 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcc248-bc07-4ab7-aac6-a952f925386c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:32.947708 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-36bcc248-bc07-4ab7-aac6-a952f925386c None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:00:32.949422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-36bcc248-bc07-4ab7-aac6-a952f925386c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:00:32.957038 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 719/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 20:00:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:00:33.108501 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c926c6-7ae4-44a1-8fd6-185358156380 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:00:33.122454 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c926c6-7ae4-44a1-8fd6-185358156380 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:00:33.127762 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dcd30b-a6aa-47de-befa-76bcacfd5dd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:00:33.128421 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dcd30b-a6aa-47de-befa-76bcacfd5dd9 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:33.128707 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dcd30b-a6aa-47de-befa-76bcacfd5dd9 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:33.128982 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dcd30b-a6aa-47de-befa-76bcacfd5dd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:33.137857 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dcd30b-a6aa-47de-befa-76bcacfd5dd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:00:33.139436 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dcd30b-a6aa-47de-befa-76bcacfd5dd9 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:00:33.148787 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 709/1428] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:00:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:00:34.083799 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62949b-3ab8-4d76-a6cc-932029bc8cfa None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:34.084451 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62949b-3ab8-4d76-a6cc-932029bc8cfa None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:34.084451 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62949b-3ab8-4d76-a6cc-932029bc8cfa None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:34.085777 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 720/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 20:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:00:34.091029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6cc06-1226-4fcc-a9fd-b7915d77af16 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:34.091437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6cc06-1226-4fcc-a9fd-b7915d77af16 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:34.091582 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6cc06-1226-4fcc-a9fd-b7915d77af16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:34.109460 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-07a6cc06-1226-4fcc-a9fd-b7915d77af16 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:00:34.110640 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-07a6cc06-1226-4fcc-a9fd-b7915d77af16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:00:34.118220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 710/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 20:00:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:00:34.265163 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62949b-3ab8-4d76-a6cc-932029bc8cfa None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:00:34.279668 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62949b-3ab8-4d76-a6cc-932029bc8cfa None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:00:34.284923 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c7d56-16b3-4dcd-9a41-e85bd3d1e0ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:00:34.285543 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c7d56-16b3-4dcd-9a41-e85bd3d1e0ee service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:34.285825 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c7d56-16b3-4dcd-9a41-e85bd3d1e0ee service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:34.286097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c7d56-16b3-4dcd-9a41-e85bd3d1e0ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:34.294553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7c7d56-16b3-4dcd-9a41-e85bd3d1e0ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:00:34.296185 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7c7d56-16b3-4dcd-9a41-e85bd3d1e0ee service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:00:34.305392 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 721/1431] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:00:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:00:35.164815 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900dfdb8-0854-454d-a7d4-f9c2cc9b7216 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:35.165235 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900dfdb8-0854-454d-a7d4-f9c2cc9b7216 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:35.165393 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900dfdb8-0854-454d-a7d4-f9c2cc9b7216 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:35.166820 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 711/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 20:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:00:35.172097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994554c3-066f-4ba4-8201-041f5d411b9f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:35.172671 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994554c3-066f-4ba4-8201-041f5d411b9f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:35.172671 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994554c3-066f-4ba4-8201-041f5d411b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:35.190053 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-994554c3-066f-4ba4-8201-041f5d411b9f None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:00:35.191235 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-994554c3-066f-4ba4-8201-041f5d411b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:00:35.199664 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 722/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 20:00:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:00:35.349679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900dfdb8-0854-454d-a7d4-f9c2cc9b7216 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:00:35.364121 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900dfdb8-0854-454d-a7d4-f9c2cc9b7216 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:00:35.369376 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ae286-8001-45ee-a42d-46dc077d5abd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:00:35.369995 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ae286-8001-45ee-a42d-46dc077d5abd service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:35.370273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ae286-8001-45ee-a42d-46dc077d5abd service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:35.370541 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ae286-8001-45ee-a42d-46dc077d5abd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:35.379114 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ae286-8001-45ee-a42d-46dc077d5abd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:00:35.380723 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ae286-8001-45ee-a42d-46dc077d5abd service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:00:35.389948 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 712/1434] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:00:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:00:36.281916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16870b-9a95-4a09-9ddf-5cb6e4f49a75 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:36.282433 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16870b-9a95-4a09-9ddf-5cb6e4f49a75 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:36.282540 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad16870b-9a95-4a09-9ddf-5cb6e4f49a75 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:36.283855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 723/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 20:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:00:36.288729 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595612e-17c2-408d-a609-b1252e9cf480 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:36.289142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595612e-17c2-408d-a609-b1252e9cf480 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:36.289382 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595612e-17c2-408d-a609-b1252e9cf480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:36.306952 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9595612e-17c2-408d-a609-b1252e9cf480 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:00:36.308636 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9595612e-17c2-408d-a609-b1252e9cf480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:00:36.316222 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 713/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 20:00:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:00:36.465556 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad16870b-9a95-4a09-9ddf-5cb6e4f49a75 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:00:36.480044 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad16870b-9a95-4a09-9ddf-5cb6e4f49a75 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:00:36.485479 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d6aec-6f38-4767-ade2-7b0bdaa55ecb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:00:36.486114 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d6aec-6f38-4767-ade2-7b0bdaa55ecb service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:36.486400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d6aec-6f38-4767-ade2-7b0bdaa55ecb service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:36.486677 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d6aec-6f38-4767-ade2-7b0bdaa55ecb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:36.494912 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d6aec-6f38-4767-ade2-7b0bdaa55ecb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:00:36.497048 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d6aec-6f38-4767-ade2-7b0bdaa55ecb service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:00:36.506407 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 724/1437] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:00:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:00:37.378700 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0353e3-6895-4669-81ee-12fa5c68f7b8 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:37.379280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0353e3-6895-4669-81ee-12fa5c68f7b8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:37.379280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0353e3-6895-4669-81ee-12fa5c68f7b8 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:37.380688 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 714/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 20:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:00:37.385862 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1302aa-c091-40c2-8bb2-2d92960a3880 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:37.386297 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1302aa-c091-40c2-8bb2-2d92960a3880 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:37.386394 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1302aa-c091-40c2-8bb2-2d92960a3880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:37.404290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ca1302aa-c091-40c2-8bb2-2d92960a3880 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:00:37.405507 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ca1302aa-c091-40c2-8bb2-2d92960a3880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:00:37.413164 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 725/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 20:00:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:00:37.562020 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0353e3-6895-4669-81ee-12fa5c68f7b8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:00:37.577030 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0353e3-6895-4669-81ee-12fa5c68f7b8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:00:37.582376 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13afcede-0abe-40a8-9aa4-4cc35d7b8dd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:00:37.583000 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13afcede-0abe-40a8-9aa4-4cc35d7b8dd3 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:37.583278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13afcede-0abe-40a8-9aa4-4cc35d7b8dd3 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:37.583548 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13afcede-0abe-40a8-9aa4-4cc35d7b8dd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:37.592000 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13afcede-0abe-40a8-9aa4-4cc35d7b8dd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:00:37.593606 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13afcede-0abe-40a8-9aa4-4cc35d7b8dd3 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:00:37.602800 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 715/1440] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:00:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:00:39.545172 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48247ba-d9bc-47a8-9ddb-b27e51cd05dc None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:39.545732 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48247ba-d9bc-47a8-9ddb-b27e51cd05dc None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:39.545732 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48247ba-d9bc-47a8-9ddb-b27e51cd05dc None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:39.546981 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 726/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 20:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:00:39.552436 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a3afe-167d-43e2-a9f2-dc8b0777e746 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:39.552841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a3afe-167d-43e2-a9f2-dc8b0777e746 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:39.552992 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a3afe-167d-43e2-a9f2-dc8b0777e746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:39.570474 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c96a3afe-167d-43e2-a9f2-dc8b0777e746 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:00:39.571960 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c96a3afe-167d-43e2-a9f2-dc8b0777e746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:00:39.579937 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 716/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 20:00:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:00:39.618622 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48247ba-d9bc-47a8-9ddb-b27e51cd05dc None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:00:39.629275 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48247ba-d9bc-47a8-9ddb-b27e51cd05dc None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:00:39.632754 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c10727e-8974-4473-8e74-0dd68cda23bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:00:39.633381 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10727e-8974-4473-8e74-0dd68cda23bb service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:39.633658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10727e-8974-4473-8e74-0dd68cda23bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:39.633929 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10727e-8974-4473-8e74-0dd68cda23bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:39.642641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c10727e-8974-4473-8e74-0dd68cda23bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:00:39.644075 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c10727e-8974-4473-8e74-0dd68cda23bb service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:00:39.653749 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 727/1443] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:00:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:00:41.866477 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21320ec0-cbdb-4e9d-bccc-8bbe5bfcca58 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:41.867059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21320ec0-cbdb-4e9d-bccc-8bbe5bfcca58 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:41.867059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21320ec0-cbdb-4e9d-bccc-8bbe5bfcca58 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:41.868441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 717/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 20:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:00:41.874390 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b620bb-0c06-451b-80ce-9f2d4e187065 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:41.874799 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b620bb-0c06-451b-80ce-9f2d4e187065 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:41.874942 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b620bb-0c06-451b-80ce-9f2d4e187065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:41.893102 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a8b620bb-0c06-451b-80ce-9f2d4e187065 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:00:41.894300 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a8b620bb-0c06-451b-80ce-9f2d4e187065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:00:41.902104 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 728/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 20:00:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:00:41.941097 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21320ec0-cbdb-4e9d-bccc-8bbe5bfcca58 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:00:41.953022 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21320ec0-cbdb-4e9d-bccc-8bbe5bfcca58 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:00:41.956765 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b8de4-401d-4b85-a165-a5da1dab9021 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:00:41.957410 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b8de4-401d-4b85-a165-a5da1dab9021 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:41.957715 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b8de4-401d-4b85-a165-a5da1dab9021 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:41.957990 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b8de4-401d-4b85-a165-a5da1dab9021 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:41.967392 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719b8de4-401d-4b85-a165-a5da1dab9021 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:00:41.970514 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719b8de4-401d-4b85-a165-a5da1dab9021 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:00:41.978367 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 718/1446] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:00:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:00:44.394190 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec11fe5-9433-4266-923e-a7911ab0062b None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:44.394719 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec11fe5-9433-4266-923e-a7911ab0062b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:44.394719 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec11fe5-9433-4266-923e-a7911ab0062b None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:44.396099 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 729/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 20:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:00:44.401376 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b55a92-75b6-464d-b008-61e22c80822f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:44.401864 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b55a92-75b6-464d-b008-61e22c80822f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:44.402168 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b55a92-75b6-464d-b008-61e22c80822f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:44.421059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b2b55a92-75b6-464d-b008-61e22c80822f None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:00:44.422259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b2b55a92-75b6-464d-b008-61e22c80822f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:00:44.429922 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 719/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 20:00:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:00:44.468127 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec11fe5-9433-4266-923e-a7911ab0062b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:00:44.478724 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec11fe5-9433-4266-923e-a7911ab0062b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:00:44.482830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66846f51-32f6-4e4d-baf1-b8a3f4a9b68d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:00:44.483465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66846f51-32f6-4e4d-baf1-b8a3f4a9b68d service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:44.483741 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66846f51-32f6-4e4d-baf1-b8a3f4a9b68d service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:44.484012 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66846f51-32f6-4e4d-baf1-b8a3f4a9b68d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:44.492912 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66846f51-32f6-4e4d-baf1-b8a3f4a9b68d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:00:44.494287 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66846f51-32f6-4e4d-baf1-b8a3f4a9b68d service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:00:44.504206 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 730/1449] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:00:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:00:46.649071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3faa0d-e32e-4767-b725-7450c840fb63 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:46.649652 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3faa0d-e32e-4767-b725-7450c840fb63 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:46.649652 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3faa0d-e32e-4767-b725-7450c840fb63 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:46.651029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 720/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 20:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:00:46.656271 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96120e9d-d981-45c4-9ffa-c520e8d696e5 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:46.656717 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96120e9d-d981-45c4-9ffa-c520e8d696e5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:46.656837 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96120e9d-d981-45c4-9ffa-c520e8d696e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:46.675310 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-96120e9d-d981-45c4-9ffa-c520e8d696e5 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:00:46.676535 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-96120e9d-d981-45c4-9ffa-c520e8d696e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:00:46.684867 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 731/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 20:00:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:00:46.723008 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3faa0d-e32e-4767-b725-7450c840fb63 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:00:46.734017 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3faa0d-e32e-4767-b725-7450c840fb63 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:00:46.737452 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a05008-c351-454d-b17a-1b06ceac78ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:00:46.738080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a05008-c351-454d-b17a-1b06ceac78ca service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:46.738362 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a05008-c351-454d-b17a-1b06ceac78ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:46.738635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a05008-c351-454d-b17a-1b06ceac78ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:46.747975 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a05008-c351-454d-b17a-1b06ceac78ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:00:46.749441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a05008-c351-454d-b17a-1b06ceac78ca service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:00:46.759177 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 721/1452] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:00:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:00:48.878393 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c5ea2-456b-4508-b030-74f890f816bf None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:48.878970 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c5ea2-456b-4508-b030-74f890f816bf None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:48.878970 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c5ea2-456b-4508-b030-74f890f816bf None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:48.880388 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 732/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 20:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:00:48.886656 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25117b-be1e-485b-9608-6ebb6034fccf None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:48.887162 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25117b-be1e-485b-9608-6ebb6034fccf None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:48.887308 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25117b-be1e-485b-9608-6ebb6034fccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:48.905857 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7c25117b-be1e-485b-9608-6ebb6034fccf None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:00:48.907049 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7c25117b-be1e-485b-9608-6ebb6034fccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:00:48.914891 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 722/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 20:00:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:00:48.953395 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c5ea2-456b-4508-b030-74f890f816bf None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:00:48.963842 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c5ea2-456b-4508-b030-74f890f816bf None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:00:48.967977 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af4e9a-010a-4f62-9ef1-d06a10e5bc7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:00:48.968635 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af4e9a-010a-4f62-9ef1-d06a10e5bc7f service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:48.968915 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af4e9a-010a-4f62-9ef1-d06a10e5bc7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:48.969183 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af4e9a-010a-4f62-9ef1-d06a10e5bc7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:48.977726 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af4e9a-010a-4f62-9ef1-d06a10e5bc7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:00:48.979156 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af4e9a-010a-4f62-9ef1-d06a10e5bc7f service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:00:48.988237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 733/1455] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:00:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:00:51.055438 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735362e6-d60c-4904-aa90-5dd3cd15a122 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:51.056009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735362e6-d60c-4904-aa90-5dd3cd15a122 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:51.056009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735362e6-d60c-4904-aa90-5dd3cd15a122 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:51.057425 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 723/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 20:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:00:51.063250 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18cbc4-e3b9-4363-8426-e9f5d29ef0e9 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:51.063682 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18cbc4-e3b9-4363-8426-e9f5d29ef0e9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:51.063803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18cbc4-e3b9-4363-8426-e9f5d29ef0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:51.081346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-eb18cbc4-e3b9-4363-8426-e9f5d29ef0e9 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:00:51.082527 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-eb18cbc4-e3b9-4363-8426-e9f5d29ef0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:00:51.090240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 734/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 20:00:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:00:51.128501 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735362e6-d60c-4904-aa90-5dd3cd15a122 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:00:51.139073 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735362e6-d60c-4904-aa90-5dd3cd15a122 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:00:51.142983 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d638d-52fd-464d-88db-dec5a2a446b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:00:51.143144 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d638d-52fd-464d-88db-dec5a2a446b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:51.143424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d638d-52fd-464d-88db-dec5a2a446b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:51.143692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d638d-52fd-464d-88db-dec5a2a446b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:51.152653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d638d-52fd-464d-88db-dec5a2a446b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:00:51.154080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d638d-52fd-464d-88db-dec5a2a446b8 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:00:51.163250 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 724/1458] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:00:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:00:53.251499 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da197dc5-1196-4946-a0a9-3ffa6cbc100c None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:53.251990 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da197dc5-1196-4946-a0a9-3ffa6cbc100c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:53.252134 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da197dc5-1196-4946-a0a9-3ffa6cbc100c None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:53.253464 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 735/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 20:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:00:53.261364 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36a290-046a-4ae5-a349-4dc7f543eb33 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:53.261851 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36a290-046a-4ae5-a349-4dc7f543eb33 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:53.261950 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36a290-046a-4ae5-a349-4dc7f543eb33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:53.279927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-2a36a290-046a-4ae5-a349-4dc7f543eb33 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:00:53.281154 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2a36a290-046a-4ae5-a349-4dc7f543eb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:00:53.290158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 725/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 20:00:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:00:53.328796 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da197dc5-1196-4946-a0a9-3ffa6cbc100c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:00:53.340648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da197dc5-1196-4946-a0a9-3ffa6cbc100c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:00:53.344084 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe3955-dc59-411a-87e9-212437c5de28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:00:53.344757 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe3955-dc59-411a-87e9-212437c5de28 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:53.345154 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe3955-dc59-411a-87e9-212437c5de28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:53.345426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe3955-dc59-411a-87e9-212437c5de28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:53.355123 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbe3955-dc59-411a-87e9-212437c5de28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:00:53.356589 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbe3955-dc59-411a-87e9-212437c5de28 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:00:53.366874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 736/1461] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:00:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:00:55.504721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282abf41-6de2-4ca6-a956-4e095984dec6 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:55.505233 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282abf41-6de2-4ca6-a956-4e095984dec6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:55.505233 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282abf41-6de2-4ca6-a956-4e095984dec6 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:55.506565 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 726/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 20:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:00:55.512360 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494419a-1122-4758-b009-5fecd03133f4 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:55.512852 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494419a-1122-4758-b009-5fecd03133f4 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:55.512958 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494419a-1122-4758-b009-5fecd03133f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:55.519225 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=79 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:00:55.532214 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f494419a-1122-4758-b009-5fecd03133f4 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:00:55.533459 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f494419a-1122-4758-b009-5fecd03133f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:00:55.541202 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 737/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 20:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:00:55.579121 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282abf41-6de2-4ca6-a956-4e095984dec6 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:00:55.584477 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=73 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:00:55.591074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282abf41-6de2-4ca6-a956-4e095984dec6 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:00:55.594696 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2e37e-7daf-4db2-a08f-b0ef631f31d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:00:55.595658 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2e37e-7daf-4db2-a08f-b0ef631f31d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:55.595658 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2e37e-7daf-4db2-a08f-b0ef631f31d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:55.595907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2e37e-7daf-4db2-a08f-b0ef631f31d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:55.604822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2e37e-7daf-4db2-a08f-b0ef631f31d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:00:55.606253 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2e37e-7daf-4db2-a08f-b0ef631f31d8 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:00:55.615875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 727/1464] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:00:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:00:57.677306 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab0f0a-fcac-429d-bbc4-b8f62567d088 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:57.677873 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab0f0a-fcac-429d-bbc4-b8f62567d088 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:57.677873 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab0f0a-fcac-429d-bbc4-b8f62567d088 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:57.679228 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 738/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 20:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 20:00:57.685095 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ec5f6-1c7f-47e3-b4a6-1c41c2778911 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:57.685550 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ec5f6-1c7f-47e3-b4a6-1c41c2778911 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:57.685651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ec5f6-1c7f-47e3-b4a6-1c41c2778911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:57.703212 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7f3ec5f6-1c7f-47e3-b4a6-1c41c2778911 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:00:57.704456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7f3ec5f6-1c7f-47e3-b4a6-1c41c2778911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:00:57.712083 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 728/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 20:00:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:00:57.749140 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab0f0a-fcac-429d-bbc4-b8f62567d088 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:00:57.759764 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab0f0a-fcac-429d-bbc4-b8f62567d088 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:00:57.763277 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef655e-592c-4e95-a490-5ad96ebc8f98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:00:57.763901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef655e-592c-4e95-a490-5ad96ebc8f98 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:57.764183 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef655e-592c-4e95-a490-5ad96ebc8f98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:57.764502 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef655e-592c-4e95-a490-5ad96ebc8f98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:57.773223 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef655e-592c-4e95-a490-5ad96ebc8f98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:00:57.774668 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef655e-592c-4e95-a490-5ad96ebc8f98 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:00:57.783912 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 739/1467] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:00:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:00:59.848740 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45d8d5-8b9d-43aa-b281-5f8c772484cc None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:59.849286 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45d8d5-8b9d-43aa-b281-5f8c772484cc None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:59.849286 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45d8d5-8b9d-43aa-b281-5f8c772484cc None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:59.850779 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 729/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 20:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:00:59.856759 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131c970-3603-48ed-8e7d-9b23d352fd36 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:59.857168 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131c970-3603-48ed-8e7d-9b23d352fd36 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:59.857309 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131c970-3603-48ed-8e7d-9b23d352fd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:59.874450 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-1131c970-3603-48ed-8e7d-9b23d352fd36 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:00:59.875638 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1131c970-3603-48ed-8e7d-9b23d352fd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:00:59.883278 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 740/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 20:00:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:00:59.919651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45d8d5-8b9d-43aa-b281-5f8c772484cc None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:00:59.930292 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45d8d5-8b9d-43aa-b281-5f8c772484cc None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:00:59.933726 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f9868c-e23d-45e5-bc06-736e97990ed4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:00:59.934346 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9868c-e23d-45e5-bc06-736e97990ed4 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:00:59.934629 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9868c-e23d-45e5-bc06-736e97990ed4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:00:59.934901 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9868c-e23d-45e5-bc06-736e97990ed4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:00:59.943571 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f9868c-e23d-45e5-bc06-736e97990ed4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:00:59.945190 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f9868c-e23d-45e5-bc06-736e97990ed4 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:00:59.955210 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 730/1470] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:00:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:01:09.869461 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=11 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:01:09.951058 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=14 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:01:19.836008 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df62efa-aee2-4952-a4ed-02adcc247bef None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:01:19.836656 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df62efa-aee2-4952-a4ed-02adcc247bef None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:01:19.836656 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df62efa-aee2-4952-a4ed-02adcc247bef None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:01:19.837940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 741/1471] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 20:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:01:19.843312 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af6dfc-939b-4d84-bc02-a529acdb7daa None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:01:19.843927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af6dfc-939b-4d84-bc02-a529acdb7daa None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:01:19.843927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af6dfc-939b-4d84-bc02-a529acdb7daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:01:19.861596 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-11af6dfc-939b-4d84-bc02-a529acdb7daa None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:01:19.862790 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-11af6dfc-939b-4d84-bc02-a529acdb7daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:01:19.871027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 731/1472] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 20:01:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:01:19.917479 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b22b2-4bb7-46e7-a770-7e6da49fd690 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:01:19.917765 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b22b2-4bb7-46e7-a770-7e6da49fd690 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:01:19.918039 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b22b2-4bb7-46e7-a770-7e6da49fd690 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:01:19.919365 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 742/1473] 10.222.0.44 () {54 vars in 962 bytes} [Thu Jun 23 20:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:01:19.923983 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9dd8c-0d7b-4615-9915-be7baca3aded None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:01:19.924271 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9dd8c-0d7b-4615-9915-be7baca3aded None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:01:19.924576 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9dd8c-0d7b-4615-9915-be7baca3aded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:01:19.943282 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e9e9dd8c-0d7b-4615-9915-be7baca3aded None None] MFA Rules not processed for user `1dd72f952df34ad8853fe84673a41f45`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:01:19.944511 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e9e9dd8c-0d7b-4615-9915-be7baca3aded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:01:19.951944 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 732/1474] 10.222.0.44 () {58 vars in 1059 bytes} [Thu Jun 23 20:01:19 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:01:19.959302 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b22b2-4bb7-46e7-a770-7e6da49fd690 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:01:19.970550 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b22b2-4bb7-46e7-a770-7e6da49fd690 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:01:19.974099 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9360c-6a68-4b03-827e-9a00ca3032c3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd72f952df34ad8853fe84673a41f45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:01:19.974753 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9360c-6a68-4b03-827e-9a00ca3032c3 service glance] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:01:19.975033 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9360c-6a68-4b03-827e-9a00ca3032c3 service glance] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:01:19.975304 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9360c-6a68-4b03-827e-9a00ca3032c3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:01:19.984672 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9360c-6a68-4b03-827e-9a00ca3032c3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:01:19.986127 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9360c-6a68-4b03-827e-9a00ca3032c3 service glance] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:01:19.996084 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 743/1475] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:01:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:01:22.112586 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843dc0a6-6764-43d1-a2e0-62c22df70208 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:01:22.113137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843dc0a6-6764-43d1-a2e0-62c22df70208 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:01:22.113137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843dc0a6-6764-43d1-a2e0-62c22df70208 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:01:22.114533 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 733/1476] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 20:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:01:22.119702 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67739be7-3b1e-49b5-982a-23afc20ae0c9 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:01:22.120148 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67739be7-3b1e-49b5-982a-23afc20ae0c9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:01:22.120247 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67739be7-3b1e-49b5-982a-23afc20ae0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:01:22.138191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-67739be7-3b1e-49b5-982a-23afc20ae0c9 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:01:22.139382 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-67739be7-3b1e-49b5-982a-23afc20ae0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:01:22.147168 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 744/1477] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 20:01:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:01:22.170633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843dc0a6-6764-43d1-a2e0-62c22df70208 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:01:22.183006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843dc0a6-6764-43d1-a2e0-62c22df70208 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:01:22.187360 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c44930-3766-482e-a74e-3fedc2336580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:01:22.188020 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c44930-3766-482e-a74e-3fedc2336580 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:01:22.188314 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c44930-3766-482e-a74e-3fedc2336580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:01:22.188609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c44930-3766-482e-a74e-3fedc2336580 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:01:22.189361 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c44930-3766-482e-a74e-3fedc2336580 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:01:22.191110 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: WARNING py.warnings [None req-70c44930-3766-482e-a74e-3fedc2336580 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 20:01:22.191110 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: warnings.warn(msg) Jun 23 20:01:22.191110 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: Jun 23 20:01:22.192253 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c44930-3766-482e-a74e-3fedc2336580 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:01:22.327185 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 734/1478] 10.222.0.44 () {56 vars in 1108 bytes} [Thu Jun 23 20:01:22 2022] GET /identity/v3/projects => generated 2553 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:01:24.377146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187bc484-52f2-4ab2-9260-9d0d50b203b3 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:01:24.377701 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187bc484-52f2-4ab2-9260-9d0d50b203b3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:01:24.377701 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187bc484-52f2-4ab2-9260-9d0d50b203b3 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:01:24.379041 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 745/1479] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 20:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:01:24.384223 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad17bd-a7ad-402f-b2cc-f00529242604 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:01:24.384691 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad17bd-a7ad-402f-b2cc-f00529242604 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:01:24.385347 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad17bd-a7ad-402f-b2cc-f00529242604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:01:24.403168 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-50ad17bd-a7ad-402f-b2cc-f00529242604 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:01:24.404394 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-50ad17bd-a7ad-402f-b2cc-f00529242604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:01:24.412124 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 735/1480] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 20:01:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:01:24.455719 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187bc484-52f2-4ab2-9260-9d0d50b203b3 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:01:24.470267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187bc484-52f2-4ab2-9260-9d0d50b203b3 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:01:24.475606 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a255b-63ab-4387-88cf-ab89a90c73c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:01:24.476227 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a255b-63ab-4387-88cf-ab89a90c73c2 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:01:24.476536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a255b-63ab-4387-88cf-ab89a90c73c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:01:24.476810 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a255b-63ab-4387-88cf-ab89a90c73c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:01:24.485177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a255b-63ab-4387-88cf-ab89a90c73c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:01:24.486846 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a255b-63ab-4387-88cf-ab89a90c73c2 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:01:24.496379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 746/1481] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:01:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:01:26.718130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf1532-23a3-4f2c-9f07-5af37eda45d6 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:01:26.718704 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf1532-23a3-4f2c-9f07-5af37eda45d6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:01:26.718704 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf1532-23a3-4f2c-9f07-5af37eda45d6 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:01:26.720035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 736/1482] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 20:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:01:26.725150 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4c23c-047b-4112-b75f-ec89236aec63 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:01:26.725579 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4c23c-047b-4112-b75f-ec89236aec63 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:01:26.725709 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4c23c-047b-4112-b75f-ec89236aec63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:01:26.744159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-5ff4c23c-047b-4112-b75f-ec89236aec63 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:01:26.745978 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5ff4c23c-047b-4112-b75f-ec89236aec63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:01:26.753581 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 747/1483] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 20:01:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:01:26.789374 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf1532-23a3-4f2c-9f07-5af37eda45d6 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:01:26.800566 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf1532-23a3-4f2c-9f07-5af37eda45d6 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:01:26.804053 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ff772-ef18-454e-9837-c707908d6d12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:01:26.810289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ff772-ef18-454e-9837-c707908d6d12 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:01:26.810289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ff772-ef18-454e-9837-c707908d6d12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:01:26.810289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ff772-ef18-454e-9837-c707908d6d12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:01:26.814503 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ff772-ef18-454e-9837-c707908d6d12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:01:26.815951 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ff772-ef18-454e-9837-c707908d6d12 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:01:26.825551 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 737/1484] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:01:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:01:28.902055 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44966e-325a-4b07-b73e-637cee574565 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:01:28.902622 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44966e-325a-4b07-b73e-637cee574565 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:01:28.902622 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44966e-325a-4b07-b73e-637cee574565 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:01:28.903965 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 748/1485] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 20:01:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:01:28.909154 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3685b5-1675-4e12-942e-ad30c5221359 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:01:28.909622 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3685b5-1675-4e12-942e-ad30c5221359 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:01:28.909728 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3685b5-1675-4e12-942e-ad30c5221359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:01:28.927567 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7a3685b5-1675-4e12-942e-ad30c5221359 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:01:28.928903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7a3685b5-1675-4e12-942e-ad30c5221359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:01:28.936644 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 738/1486] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 20:01:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:01:28.974737 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca44966e-325a-4b07-b73e-637cee574565 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:01:28.985855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca44966e-325a-4b07-b73e-637cee574565 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:01:28.989552 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca940b1f-c4a7-4edd-a9ae-56592efde29e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:01:28.990214 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca940b1f-c4a7-4edd-a9ae-56592efde29e service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:01:28.990494 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca940b1f-c4a7-4edd-a9ae-56592efde29e service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:01:28.990765 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca940b1f-c4a7-4edd-a9ae-56592efde29e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:01:28.999718 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca940b1f-c4a7-4edd-a9ae-56592efde29e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:01:29.001183 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca940b1f-c4a7-4edd-a9ae-56592efde29e service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:01:29.011048 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 749/1487] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:01:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:01:31.125444 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff02aec-d043-4df9-8d8d-141326780af6 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:01:31.126025 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff02aec-d043-4df9-8d8d-141326780af6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:01:31.126025 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff02aec-d043-4df9-8d8d-141326780af6 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:01:31.127352 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 739/1488] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 20:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:01:31.132634 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db962a-a215-4d48-8618-08bd47b83a1e None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:01:31.133051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db962a-a215-4d48-8618-08bd47b83a1e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:01:31.133181 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db962a-a215-4d48-8618-08bd47b83a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:01:31.150352 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-25db962a-a215-4d48-8618-08bd47b83a1e None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:01:31.151529 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-25db962a-a215-4d48-8618-08bd47b83a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:01:31.159458 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 750/1489] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 20:01:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:01:31.200110 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff02aec-d043-4df9-8d8d-141326780af6 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:01:31.210728 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff02aec-d043-4df9-8d8d-141326780af6 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:01:31.214231 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36085e7-5612-4730-9bbd-d0f270ee3e00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:01:31.214853 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36085e7-5612-4730-9bbd-d0f270ee3e00 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:01:31.215132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36085e7-5612-4730-9bbd-d0f270ee3e00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:01:31.215402 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36085e7-5612-4730-9bbd-d0f270ee3e00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:01:31.224024 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36085e7-5612-4730-9bbd-d0f270ee3e00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:01:31.225482 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36085e7-5612-4730-9bbd-d0f270ee3e00 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:01:31.235420 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 740/1490] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:01:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:01:33.621696 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e2180-71d1-43c7-b3e8-c95ddb954af1 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:01:33.622234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e2180-71d1-43c7-b3e8-c95ddb954af1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:01:33.622234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e2180-71d1-43c7-b3e8-c95ddb954af1 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:01:33.623593 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 751/1491] 10.222.0.44 () {54 vars in 926 bytes} [Thu Jun 23 20:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:01:33.628876 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb03eb-0dc1-4396-9e72-fcd0f4f9cc4e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:01:33.629309 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb03eb-0dc1-4396-9e72-fcd0f4f9cc4e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:01:33.629433 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb03eb-0dc1-4396-9e72-fcd0f4f9cc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:01:33.646952 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9feb03eb-0dc1-4396-9e72-fcd0f4f9cc4e None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:01:33.648126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9feb03eb-0dc1-4396-9e72-fcd0f4f9cc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:01:33.656414 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 741/1492] 10.222.0.44 () {58 vars in 1023 bytes} [Thu Jun 23 20:01:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:01:33.694613 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e2180-71d1-43c7-b3e8-c95ddb954af1 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:01:33.705875 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e2180-71d1-43c7-b3e8-c95ddb954af1 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:01:33.709291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e2c39-5b73-4424-93a2-8edda992313d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:01:33.709908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e2c39-5b73-4424-93a2-8edda992313d service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:01:33.710184 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e2c39-5b73-4424-93a2-8edda992313d service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:01:33.710454 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e2c39-5b73-4424-93a2-8edda992313d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:01:33.719278 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e2c39-5b73-4424-93a2-8edda992313d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:01:33.720782 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e2c39-5b73-4424-93a2-8edda992313d service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:01:33.730165 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 752/1493] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:01:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:01:43.642480 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=33 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:01:43.725937 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=33 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:03:22.760362 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be8c83-bcd1-4473-ba8a-f54ea1b86d28 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:03:22.760961 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be8c83-bcd1-4473-ba8a-f54ea1b86d28 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:03:22.760961 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be8c83-bcd1-4473-ba8a-f54ea1b86d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:03:22.778305 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e2be8c83-bcd1-4473-ba8a-f54ea1b86d28 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:03:22.779515 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e2be8c83-bcd1-4473-ba8a-f54ea1b86d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:03:22.787169 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 742/1494] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:03:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:03:22.796791 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1aa34-9bee-421b-970a-74db86978806 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:03:22.797295 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1aa34-9bee-421b-970a-74db86978806 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:03:22.797404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1aa34-9bee-421b-970a-74db86978806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:03:22.815483 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-fca1aa34-9bee-421b-970a-74db86978806 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:03:22.816734 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fca1aa34-9bee-421b-970a-74db86978806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:03:22.824658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 753/1495] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:03:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:03:22.830636 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be8c83-bcd1-4473-ba8a-f54ea1b86d28 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:03:22.840280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be8c83-bcd1-4473-ba8a-f54ea1b86d28 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:03:22.845455 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30227ed9-822e-409d-b142-46eba7007ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:03:22.846121 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30227ed9-822e-409d-b142-46eba7007ddd admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:03:22.846419 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30227ed9-822e-409d-b142-46eba7007ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:03:22.846695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30227ed9-822e-409d-b142-46eba7007ddd admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:03:22.847441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30227ed9-822e-409d-b142-46eba7007ddd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:03:22.847747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30227ed9-822e-409d-b142-46eba7007ddd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:03:22.848801 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: WARNING py.warnings [None req-30227ed9-822e-409d-b142-46eba7007ddd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 20:03:22.848801 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: warnings.warn(msg) Jun 23 20:03:22.848801 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: Jun 23 20:03:22.856778 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30227ed9-822e-409d-b142-46eba7007ddd admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:03:22.862404 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 743/1496] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 20:03:22.870688 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1aa34-9bee-421b-970a-74db86978806 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:03:22.882835 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1aa34-9bee-421b-970a-74db86978806 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:03:22.887354 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0deb5-1491-42bb-afeb-69c8373851e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:03:22.888006 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0deb5-1491-42bb-afeb-69c8373851e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:03:22.888288 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0deb5-1491-42bb-afeb-69c8373851e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:03:22.888587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0deb5-1491-42bb-afeb-69c8373851e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:03:22.889285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c0deb5-1491-42bb-afeb-69c8373851e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:03:22.891029 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: WARNING py.warnings [None req-a5c0deb5-1491-42bb-afeb-69c8373851e2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 20:03:22.891029 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: warnings.warn(msg) Jun 23 20:03:22.891029 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: Jun 23 20:03:22.892219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c0deb5-1491-42bb-afeb-69c8373851e2 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:03:22.988684 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 754/1497] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:03:22 2022] POST /identity/v3/projects => generated 379 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:03:22.993058 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30227ed9-822e-409d-b142-46eba7007ddd admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:03:23.004517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30227ed9-822e-409d-b142-46eba7007ddd admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:03:23.061014 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1a675-2641-43ef-b40b-73f025828e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:03:23.061595 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1a675-2641-43ef-b40b-73f025828e3c admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:03:23.061881 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1a675-2641-43ef-b40b-73f025828e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:03:23.062151 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1a675-2641-43ef-b40b-73f025828e3c admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:03:23.062845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca1a675-2641-43ef-b40b-73f025828e3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:03:23.063879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: WARNING py.warnings [None req-3ca1a675-2641-43ef-b40b-73f025828e3c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 20:03:23.063879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: warnings.warn(msg) Jun 23 20:03:23.063879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: Jun 23 20:03:23.065132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca1a675-2641-43ef-b40b-73f025828e3c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:03:23.177724 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 744/1498] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:03:22 2022] POST /identity/v3/users => generated 290 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:03:23.181860 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0deb5-1491-42bb-afeb-69c8373851e2 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:03:23.192747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0deb5-1491-42bb-afeb-69c8373851e2 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:03:23.197518 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272aed82-2c9e-4a9c-b5ac-261138956667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:03:23.198177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272aed82-2c9e-4a9c-b5ac-261138956667 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:03:23.198459 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272aed82-2c9e-4a9c-b5ac-261138956667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:03:23.198736 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272aed82-2c9e-4a9c-b5ac-261138956667 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:03:23.199463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272aed82-2c9e-4a9c-b5ac-261138956667 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:03:23.200524 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: WARNING py.warnings [None req-272aed82-2c9e-4a9c-b5ac-261138956667 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 20:03:23.200524 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: warnings.warn(msg) Jun 23 20:03:23.200524 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: Jun 23 20:03:23.201995 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272aed82-2c9e-4a9c-b5ac-261138956667 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:03:23.214434 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 755/1499] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:03:23.218376 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1a675-2641-43ef-b40b-73f025828e3c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:03:23.228020 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1a675-2641-43ef-b40b-73f025828e3c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:03:23.232714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa0aa6-7a26-49b0-bb17-41853821496a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:03:23.233458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa0aa6-7a26-49b0-bb17-41853821496a admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:03:23.233849 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa0aa6-7a26-49b0-bb17-41853821496a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:03:23.234138 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa0aa6-7a26-49b0-bb17-41853821496a admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:03:23.234898 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa0aa6-7a26-49b0-bb17-41853821496a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:03:23.235878 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: WARNING py.warnings [None req-25fa0aa6-7a26-49b0-bb17-41853821496a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 20:03:23.235878 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: warnings.warn(msg) Jun 23 20:03:23.235878 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: Jun 23 20:03:23.236828 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa0aa6-7a26-49b0-bb17-41853821496a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:03:23.250894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 745/1500] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:03:23.255013 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272aed82-2c9e-4a9c-b5ac-261138956667 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:03:23.265325 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272aed82-2c9e-4a9c-b5ac-261138956667 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:03:23.269803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac2735-84e2-46b4-a0e0-9954f9f19b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:03:23.270670 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac2735-84e2-46b4-a0e0-9954f9f19b9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:03:23.270953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac2735-84e2-46b4-a0e0-9954f9f19b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:03:23.271226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac2735-84e2-46b4-a0e0-9954f9f19b9e admin admin] PATH_INFO: `/v3/projects/e8060af9c77b4025bffffa12a8b351ce/users/32c348326eeb4b89a24aa396a423e7e6/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:03:23.286313 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac2735-84e2-46b4-a0e0-9954f9f19b9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8060af9c77b4025bffffa12a8b351ce, user_id=32c348326eeb4b89a24aa396a423e7e6, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:03:23.287354 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: WARNING py.warnings [None req-aaac2735-84e2-46b4-a0e0-9954f9f19b9e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 20:03:23.287354 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: warnings.warn(msg) Jun 23 20:03:23.287354 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: Jun 23 20:03:23.289378 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac2735-84e2-46b4-a0e0-9954f9f19b9e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:03:23.372097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 756/1501] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:03:23 2022] PUT /identity/v3/projects/e8060af9c77b4025bffffa12a8b351ce/users/32c348326eeb4b89a24aa396a423e7e6/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:03:23.384044 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccc103-bca5-42cd-8712-0c64a59ff5f2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:03:23.384365 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccc103-bca5-42cd-8712-0c64a59ff5f2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:03:23.384692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ccc103-bca5-42cd-8712-0c64a59ff5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:03:23.412721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c8ccc103-bca5-42cd-8712-0c64a59ff5f2 None None] MFA Rules not processed for user `32c348326eeb4b89a24aa396a423e7e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:03:23.413938 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c8ccc103-bca5-42cd-8712-0c64a59ff5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:03:23.480183 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 746/1502] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:03:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:03:23.493440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0b31b-44e9-47d7-9351-8befb6188cbf None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:03:23.493726 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0b31b-44e9-47d7-9351-8befb6188cbf None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:03:23.494001 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0b31b-44e9-47d7-9351-8befb6188cbf None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:03:23.495358 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 757/1503] 10.222.0.44 () {54 vars in 963 bytes} [Thu Jun 23 20:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:03:23.500375 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33160543-5c02-4c2c-b77c-c15ef18a0907 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:03:23.537059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33160543-5c02-4c2c-b77c-c15ef18a0907 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:03:23.537059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33160543-5c02-4c2c-b77c-c15ef18a0907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:03:23.537059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-33160543-5c02-4c2c-b77c-c15ef18a0907 None None] MFA Rules not processed for user `21ae3b8971b44d928e9f6c0539858636`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:03:23.537059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-33160543-5c02-4c2c-b77c-c15ef18a0907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:03:23.562960 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:03:23.570186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 747/1504] 10.222.0.44 () {58 vars in 1060 bytes} [Thu Jun 23 20:03:23 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:03:23.578565 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0b31b-44e9-47d7-9351-8befb6188cbf None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:03:23.589005 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0b31b-44e9-47d7-9351-8befb6188cbf None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:03:23.593018 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e8e80-d76e-435e-b203-b8cb54b1a551 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:03:23.593638 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e8e80-d76e-435e-b203-b8cb54b1a551 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:03:23.593914 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e8e80-d76e-435e-b203-b8cb54b1a551 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:03:23.594181 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e8e80-d76e-435e-b203-b8cb54b1a551 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:03:23.602597 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e8e80-d76e-435e-b203-b8cb54b1a551 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:03:23.604069 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e8e80-d76e-435e-b203-b8cb54b1a551 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:03:23.613096 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 758/1505] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:03:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:03:23.659876 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88d9cf-e9bc-4e91-8fb3-d342802da0d0 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:03:23.660162 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88d9cf-e9bc-4e91-8fb3-d342802da0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:03:23.660468 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88d9cf-e9bc-4e91-8fb3-d342802da0d0 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:03:23.661760 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 748/1506] 10.222.0.44 () {52 vars in 829 bytes} [Thu Jun 23 20:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:03:23.665289 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e8e80-d76e-435e-b203-b8cb54b1a551 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:03:23.725700 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e8e80-d76e-435e-b203-b8cb54b1a551 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:03:23.730116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e4b1d-8ff2-493e-9e0a-3b4bd36057ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:03:23.730962 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e4b1d-8ff2-493e-9e0a-3b4bd36057ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:03:23.731236 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e4b1d-8ff2-493e-9e0a-3b4bd36057ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:03:23.731502 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e4b1d-8ff2-493e-9e0a-3b4bd36057ec admin admin] PATH_INFO: `/v3/users/32c348326eeb4b89a24aa396a423e7e6` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:03:23.732769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e4b1d-8ff2-493e-9e0a-3b4bd36057ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=32c348326eeb4b89a24aa396a423e7e6)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:03:23.733785 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: WARNING py.warnings [None req-071e4b1d-8ff2-493e-9e0a-3b4bd36057ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 20:03:23.733785 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: warnings.warn(msg) Jun 23 20:03:23.733785 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: Jun 23 20:03:23.734976 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e4b1d-8ff2-493e-9e0a-3b4bd36057ec admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:03:23.943979 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-071e4b1d-8ff2-493e-9e0a-3b4bd36057ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32c348326eeb4b89a24aa396a423e7e6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:03:23.944665 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-071e4b1d-8ff2-493e-9e0a-3b4bd36057ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32c348326eeb4b89a24aa396a423e7e6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:03:24.011298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-071e4b1d-8ff2-493e-9e0a-3b4bd36057ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32c348326eeb4b89a24aa396a423e7e6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:03:24.011949 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-071e4b1d-8ff2-493e-9e0a-3b4bd36057ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32c348326eeb4b89a24aa396a423e7e6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:03:24.030801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-071e4b1d-8ff2-493e-9e0a-3b4bd36057ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32c348326eeb4b89a24aa396a423e7e6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:03:24.039101 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:03:24.048513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 759/1507] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:03:23 2022] DELETE /identity/v3/users/32c348326eeb4b89a24aa396a423e7e6 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:03:24.052935 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650c7aa-40e9-4423-8d75-2ba0a1ff3061 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:03:24.053401 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650c7aa-40e9-4423-8d75-2ba0a1ff3061 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:03:24.053508 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650c7aa-40e9-4423-8d75-2ba0a1ff3061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:03:24.072804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f650c7aa-40e9-4423-8d75-2ba0a1ff3061 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:03:24.075081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f650c7aa-40e9-4423-8d75-2ba0a1ff3061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:03:24.132132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 749/1508] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:03:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:03:24.145554 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e4b1d-8ff2-493e-9e0a-3b4bd36057ec admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:03:24.146973 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-071e4b1d-8ff2-493e-9e0a-3b4bd36057ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:03:24.204483 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e4b1d-8ff2-493e-9e0a-3b4bd36057ec admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:03:24.207898 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602666d-788f-4104-88ab-4d6ad281d69e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:03:24.208565 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602666d-788f-4104-88ab-4d6ad281d69e service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:03:24.208850 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602666d-788f-4104-88ab-4d6ad281d69e service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:03:24.209131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602666d-788f-4104-88ab-4d6ad281d69e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:03:24.217825 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602666d-788f-4104-88ab-4d6ad281d69e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:03:24.219266 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602666d-788f-4104-88ab-4d6ad281d69e service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:03:24.228716 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 760/1509] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:03:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:03:24.693285 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650c7aa-40e9-4423-8d75-2ba0a1ff3061 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:03:24.694711 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f650c7aa-40e9-4423-8d75-2ba0a1ff3061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:03:24.706839 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650c7aa-40e9-4423-8d75-2ba0a1ff3061 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:03:24.711638 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb422bd2-6b42-4c2c-bd34-22ef9a88b4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:03:24.713865 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb422bd2-6b42-4c2c-bd34-22ef9a88b4b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:03:24.714149 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb422bd2-6b42-4c2c-bd34-22ef9a88b4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:03:24.714437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb422bd2-6b42-4c2c-bd34-22ef9a88b4b2 admin admin] PATH_INFO: `/v3/projects/e8060af9c77b4025bffffa12a8b351ce` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:03:24.715723 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb422bd2-6b42-4c2c-bd34-22ef9a88b4b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8060af9c77b4025bffffa12a8b351ce)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:03:24.716669 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: WARNING py.warnings [None req-cb422bd2-6b42-4c2c-bd34-22ef9a88b4b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 20:03:24.716669 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: warnings.warn(msg) Jun 23 20:03:24.716669 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: Jun 23 20:03:24.719629 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb422bd2-6b42-4c2c-bd34-22ef9a88b4b2 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:03:24.847750 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cb422bd2-6b42-4c2c-bd34-22ef9a88b4b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8060af9c77b4025bffffa12a8b351ce', 'request_id': 'req-cb422bd2-6b42-4c2c-bd34-22ef9a88b4b2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:03:24.861422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cb422bd2-6b42-4c2c-bd34-22ef9a88b4b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8060af9c77b4025bffffa12a8b351ce', 'request_id': 'req-cb422bd2-6b42-4c2c-bd34-22ef9a88b4b2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:03:24.867370 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cb422bd2-6b42-4c2c-bd34-22ef9a88b4b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8060af9c77b4025bffffa12a8b351ce', 'request_id': 'req-cb422bd2-6b42-4c2c-bd34-22ef9a88b4b2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:03:24.941907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cb422bd2-6b42-4c2c-bd34-22ef9a88b4b2 admin admin] The token cache is being invalidate because project e8060af9c77b4025bffffa12a8b351ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:03:24.942192 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cb422bd2-6b42-4c2c-bd34-22ef9a88b4b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:03:24.942807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cb422bd2-6b42-4c2c-bd34-22ef9a88b4b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:03:24.944478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 750/1510] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:03:24 2022] DELETE /identity/v3/projects/e8060af9c77b4025bffffa12a8b351ce => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:03:34.224440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=29 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:03:34.938800 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:05:01.967995 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2e029-d726-4603-93fb-dd5b794980a7 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:01.968608 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2e029-d726-4603-93fb-dd5b794980a7 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:01.968715 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2e029-d726-4603-93fb-dd5b794980a7 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:01.970005 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 761/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 20:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:05:01.974821 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda46b8-a975-4747-a38f-8377a6fe54f7 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:01.975237 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda46b8-a975-4747-a38f-8377a6fe54f7 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:01.975382 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda46b8-a975-4747-a38f-8377a6fe54f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:01.994086 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-2fda46b8-a975-4747-a38f-8377a6fe54f7 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:05:01.995987 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2fda46b8-a975-4747-a38f-8377a6fe54f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:05:02.055621 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 751/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 20:05:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:05:02.208737 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2e029-d726-4603-93fb-dd5b794980a7 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:05:02.210465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c9a2e029-d726-4603-93fb-dd5b794980a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:05:02.285727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2e029-d726-4603-93fb-dd5b794980a7 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:05:02.291031 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730c4b7-a3d6-41cf-b864-97617b1cfe57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:05:02.291673 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730c4b7-a3d6-41cf-b864-97617b1cfe57 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:02.291953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730c4b7-a3d6-41cf-b864-97617b1cfe57 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:02.292252 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730c4b7-a3d6-41cf-b864-97617b1cfe57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:02.300691 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730c4b7-a3d6-41cf-b864-97617b1cfe57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:05:02.302307 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730c4b7-a3d6-41cf-b864-97617b1cfe57 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:05:02.311564 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 762/1513] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:05:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:05:03.144350 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc15c1a-12c5-4d53-b2ae-f76171d3b96d None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:03.144923 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc15c1a-12c5-4d53-b2ae-f76171d3b96d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:03.144923 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc15c1a-12c5-4d53-b2ae-f76171d3b96d None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:03.146294 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 752/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 20:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:05:03.150943 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35859899-b2a4-4ce2-897f-83425ed8a061 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:03.151361 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35859899-b2a4-4ce2-897f-83425ed8a061 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:03.151506 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35859899-b2a4-4ce2-897f-83425ed8a061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:03.169868 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-35859899-b2a4-4ce2-897f-83425ed8a061 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:05:03.171070 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-35859899-b2a4-4ce2-897f-83425ed8a061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:05:03.178775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 763/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 20:05:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:05:03.335287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc15c1a-12c5-4d53-b2ae-f76171d3b96d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:05:03.350406 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc15c1a-12c5-4d53-b2ae-f76171d3b96d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:05:03.356062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee78db26-b8be-4a57-aba9-396a8db1e68d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:05:03.356715 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78db26-b8be-4a57-aba9-396a8db1e68d service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:03.356998 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78db26-b8be-4a57-aba9-396a8db1e68d service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:03.357270 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78db26-b8be-4a57-aba9-396a8db1e68d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:03.366125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee78db26-b8be-4a57-aba9-396a8db1e68d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:05:03.367708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee78db26-b8be-4a57-aba9-396a8db1e68d service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:05:03.377013 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 753/1516] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:05:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:05:03.461866 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbacc2d-61b3-4370-a97b-eca77c8c5142 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:03.462189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbacc2d-61b3-4370-a97b-eca77c8c5142 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:03.462472 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbacc2d-61b3-4370-a97b-eca77c8c5142 None None] PATH_INFO: `/v3` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:03.463811 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 764/1517] 10.222.0.42 () {54 vars in 925 bytes} [Thu Jun 23 20:05:03 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:05:03.470066 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3b4ed-f74a-41ae-8c4f-4dc8ff6c6878 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:03.470357 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3b4ed-f74a-41ae-8c4f-4dc8ff6c6878 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:03.470653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3b4ed-f74a-41ae-8c4f-4dc8ff6c6878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:03.488792 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-69f3b4ed-f74a-41ae-8c4f-4dc8ff6c6878 None None] MFA Rules not processed for user `9715d5deeccd419b97cb0bd39e17982f`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:05:03.489992 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-69f3b4ed-f74a-41ae-8c4f-4dc8ff6c6878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:05:03.550851 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 754/1518] 10.222.0.42 () {58 vars in 1015 bytes} [Thu Jun 23 20:05:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:05:12.697321 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f0710-1e9b-4ae2-adc8-e5d5c03a3e2d None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:12.697932 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f0710-1e9b-4ae2-adc8-e5d5c03a3e2d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:12.697932 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f0710-1e9b-4ae2-adc8-e5d5c03a3e2d None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:12.699267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 765/1519] 10.222.0.44 () {54 vars in 960 bytes} [Thu Jun 23 20:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:05:12.705080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752fcbd-907d-4552-9ded-20ea81256d42 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:12.705507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752fcbd-907d-4552-9ded-20ea81256d42 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:12.705633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752fcbd-907d-4552-9ded-20ea81256d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:12.723418 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-3752fcbd-907d-4552-9ded-20ea81256d42 None None] MFA Rules not processed for user `9715d5deeccd419b97cb0bd39e17982f`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:05:12.724663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3752fcbd-907d-4552-9ded-20ea81256d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:05:12.733345 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 755/1520] 10.222.0.44 () {58 vars in 1057 bytes} [Thu Jun 23 20:05:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:05:12.740847 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f0710-1e9b-4ae2-adc8-e5d5c03a3e2d None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:05:12.754411 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f0710-1e9b-4ae2-adc8-e5d5c03a3e2d None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:05:12.759177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbc0b4-9f9e-4a53-9444-1f7f83625966 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9715d5deeccd419b97cb0bd39e17982f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:05:12.759793 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbc0b4-9f9e-4a53-9444-1f7f83625966 service placement] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:12.760093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbc0b4-9f9e-4a53-9444-1f7f83625966 service placement] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:12.760392 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbc0b4-9f9e-4a53-9444-1f7f83625966 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:12.769105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbc0b4-9f9e-4a53-9444-1f7f83625966 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:05:12.770677 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbc0b4-9f9e-4a53-9444-1f7f83625966 service placement] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:05:12.780050 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 766/1521] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:05:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:05:14.601150 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c39cc-52df-4d3f-8be8-f7308cf02caf None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:14.601704 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c39cc-52df-4d3f-8be8-f7308cf02caf None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:14.601704 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c39cc-52df-4d3f-8be8-f7308cf02caf None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:14.603075 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 756/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 20:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:05:14.607847 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf234d-5f80-4f66-bc2a-c783d7863e60 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:14.608276 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf234d-5f80-4f66-bc2a-c783d7863e60 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:14.608405 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf234d-5f80-4f66-bc2a-c783d7863e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:14.627031 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9bdf234d-5f80-4f66-bc2a-c783d7863e60 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:05:14.628220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9bdf234d-5f80-4f66-bc2a-c783d7863e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:05:14.635936 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 767/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 20:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:05:14.788321 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c39cc-52df-4d3f-8be8-f7308cf02caf None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:05:14.803755 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c39cc-52df-4d3f-8be8-f7308cf02caf None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:05:14.809101 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860cbede-0cca-4075-beb5-3f6db30f40b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:05:14.809727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cbede-0cca-4075-beb5-3f6db30f40b8 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:14.810006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cbede-0cca-4075-beb5-3f6db30f40b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:14.810277 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cbede-0cca-4075-beb5-3f6db30f40b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:14.818344 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860cbede-0cca-4075-beb5-3f6db30f40b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:05:14.819960 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860cbede-0cca-4075-beb5-3f6db30f40b8 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:05:14.829580 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 757/1524] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:05:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:05:15.651079 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5da65-9c16-4795-b293-b9b9d19c9f83 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:15.651649 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5da65-9c16-4795-b293-b9b9d19c9f83 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:15.651649 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5da65-9c16-4795-b293-b9b9d19c9f83 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:15.653106 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 768/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 20:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:05:15.657943 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddba0f8-3734-498b-8e64-59b6ab335551 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:15.658367 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddba0f8-3734-498b-8e64-59b6ab335551 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:15.658500 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddba0f8-3734-498b-8e64-59b6ab335551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:15.675971 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-cddba0f8-3734-498b-8e64-59b6ab335551 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:05:15.677223 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-cddba0f8-3734-498b-8e64-59b6ab335551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:05:15.684977 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 758/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 20:05:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:05:15.836407 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5da65-9c16-4795-b293-b9b9d19c9f83 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:05:15.850900 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5da65-9c16-4795-b293-b9b9d19c9f83 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:05:15.856213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6355f0-10c5-4246-917d-c59f0d259eae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:05:15.856868 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6355f0-10c5-4246-917d-c59f0d259eae service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:15.857147 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6355f0-10c5-4246-917d-c59f0d259eae service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:15.857420 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6355f0-10c5-4246-917d-c59f0d259eae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:15.865536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6355f0-10c5-4246-917d-c59f0d259eae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:05:15.867126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6355f0-10c5-4246-917d-c59f0d259eae service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:05:15.877280 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 769/1527] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:05:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:05:19.850951 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63436a26-af9b-4edb-996e-ac7b1755c7f9 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:19.851506 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63436a26-af9b-4edb-996e-ac7b1755c7f9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:19.851506 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63436a26-af9b-4edb-996e-ac7b1755c7f9 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:19.852921 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 759/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 20:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:05:19.858372 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbfcd73-1a3f-449d-b850-3f0ec5120e48 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:19.858797 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbfcd73-1a3f-449d-b850-3f0ec5120e48 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:19.858955 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbfcd73-1a3f-449d-b850-3f0ec5120e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:19.876762 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-abbfcd73-1a3f-449d-b850-3f0ec5120e48 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:05:19.877987 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-abbfcd73-1a3f-449d-b850-3f0ec5120e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:05:19.885719 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 770/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 20:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:05:19.922640 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63436a26-af9b-4edb-996e-ac7b1755c7f9 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:05:19.924290 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-63436a26-af9b-4edb-996e-ac7b1755c7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:05:19.982349 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63436a26-af9b-4edb-996e-ac7b1755c7f9 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:05:19.985925 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35156fe7-02bc-414b-b217-b0aaa587b31a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:05:19.986567 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35156fe7-02bc-414b-b217-b0aaa587b31a service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:19.986850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35156fe7-02bc-414b-b217-b0aaa587b31a service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:19.987127 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35156fe7-02bc-414b-b217-b0aaa587b31a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:19.996038 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35156fe7-02bc-414b-b217-b0aaa587b31a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:05:19.997511 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35156fe7-02bc-414b-b217-b0aaa587b31a service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:05:20.002413 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:05:20.007055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 760/1530] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:05:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:05:22.051752 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff4676-b3d7-4f31-9217-134b4a323368 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:22.052350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff4676-b3d7-4f31-9217-134b4a323368 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:22.052350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff4676-b3d7-4f31-9217-134b4a323368 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:22.053722 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 771/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 20:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:05:22.058769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b064a-c660-4fc5-8027-245cec263c7c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:22.059177 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b064a-c660-4fc5-8027-245cec263c7c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:22.059329 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b064a-c660-4fc5-8027-245cec263c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:22.077862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7f4b064a-c660-4fc5-8027-245cec263c7c None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:05:22.079051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7f4b064a-c660-4fc5-8027-245cec263c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:05:22.087549 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 761/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 20:05:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:05:22.123228 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff4676-b3d7-4f31-9217-134b4a323368 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:05:22.134331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff4676-b3d7-4f31-9217-134b4a323368 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:05:22.138022 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599d37f-8d27-4f40-9a04-8f7c6bde26d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:05:22.138675 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599d37f-8d27-4f40-9a04-8f7c6bde26d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:22.138957 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599d37f-8d27-4f40-9a04-8f7c6bde26d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:22.139227 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599d37f-8d27-4f40-9a04-8f7c6bde26d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:22.148031 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599d37f-8d27-4f40-9a04-8f7c6bde26d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:05:22.149692 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599d37f-8d27-4f40-9a04-8f7c6bde26d6 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:05:22.159718 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 772/1533] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:05:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:05:25.707362 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497d9b9-a31f-46ef-a8e7-8fba30ed8064 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:25.707911 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497d9b9-a31f-46ef-a8e7-8fba30ed8064 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:25.707911 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9497d9b9-a31f-46ef-a8e7-8fba30ed8064 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:25.709308 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 762/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 20:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:05:25.714565 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7b00d-81b1-43d9-af92-a77dfd82aa82 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:25.715022 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7b00d-81b1-43d9-af92-a77dfd82aa82 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:25.715129 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7b00d-81b1-43d9-af92-a77dfd82aa82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:25.721997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=62 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:05:25.734116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d1f7b00d-81b1-43d9-af92-a77dfd82aa82 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:05:25.747953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d1f7b00d-81b1-43d9-af92-a77dfd82aa82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:05:25.747953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 773/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 20:05:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:05:26.046612 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497d9b9-a31f-46ef-a8e7-8fba30ed8064 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:05:26.048276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9497d9b9-a31f-46ef-a8e7-8fba30ed8064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:05:26.105401 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9497d9b9-a31f-46ef-a8e7-8fba30ed8064 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:05:26.108819 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c68d3-f58b-4ef7-8372-994feb5e4ef5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:05:26.109449 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c68d3-f58b-4ef7-8372-994feb5e4ef5 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:26.109759 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c68d3-f58b-4ef7-8372-994feb5e4ef5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:26.110027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c68d3-f58b-4ef7-8372-994feb5e4ef5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:26.119177 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c68d3-f58b-4ef7-8372-994feb5e4ef5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:05:26.120639 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c68d3-f58b-4ef7-8372-994feb5e4ef5 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:05:26.130405 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 763/1536] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:05:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:05:28.165186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19a243-108c-42ab-9d17-a92199c505d1 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:28.165762 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19a243-108c-42ab-9d17-a92199c505d1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:28.165762 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19a243-108c-42ab-9d17-a92199c505d1 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:28.167100 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 774/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 20:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:05:28.172114 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5a4a6-2eb7-4354-bcb5-dcd50a5ade9f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:28.172542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5a4a6-2eb7-4354-bcb5-dcd50a5ade9f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:28.172691 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5a4a6-2eb7-4354-bcb5-dcd50a5ade9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:28.190298 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4bc5a4a6-2eb7-4354-bcb5-dcd50a5ade9f None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:05:28.191409 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4bc5a4a6-2eb7-4354-bcb5-dcd50a5ade9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:05:28.199318 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 764/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 20:05:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:05:28.497519 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc19a243-108c-42ab-9d17-a92199c505d1 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:05:28.499300 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fc19a243-108c-42ab-9d17-a92199c505d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:05:28.512553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc19a243-108c-42ab-9d17-a92199c505d1 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:05:28.516013 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca3e33-2c8c-49f9-8d81-e28a37f25563 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:05:28.516672 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca3e33-2c8c-49f9-8d81-e28a37f25563 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:05:28.516956 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca3e33-2c8c-49f9-8d81-e28a37f25563 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:05:28.517229 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca3e33-2c8c-49f9-8d81-e28a37f25563 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:05:28.526646 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca3e33-2c8c-49f9-8d81-e28a37f25563 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:05:28.528071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca3e33-2c8c-49f9-8d81-e28a37f25563 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:05:28.537422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 775/1539] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:05:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:05:38.185281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=32 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:05:38.533276 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=8 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:06:55.994442 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224a73f-925e-433d-a743-dc4fe246e243 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:56.029209 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224a73f-925e-433d-a743-dc4fe246e243 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:56.029209 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224a73f-925e-433d-a743-dc4fe246e243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:56.029209 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a224a73f-925e-433d-a743-dc4fe246e243 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:06:56.029209 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a224a73f-925e-433d-a743-dc4fe246e243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:06:56.030687 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 765/1540] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:06:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:06:56.040840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b85a0-c034-4bab-9716-c2f279fdeb7a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:56.041400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b85a0-c034-4bab-9716-c2f279fdeb7a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:56.041400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b85a0-c034-4bab-9716-c2f279fdeb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:56.069234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8e2b85a0-c034-4bab-9716-c2f279fdeb7a None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:06:56.070447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8e2b85a0-c034-4bab-9716-c2f279fdeb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:06:56.080046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 776/1541] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:06:56.084710 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224a73f-925e-433d-a743-dc4fe246e243 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:56.094971 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224a73f-925e-433d-a743-dc4fe246e243 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:56.108540 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0c734-8e86-4f4f-aba1-8c6aadbf9bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:56.109218 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0c734-8e86-4f4f-aba1-8c6aadbf9bba admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:56.109506 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0c734-8e86-4f4f-aba1-8c6aadbf9bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:56.109784 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0c734-8e86-4f4f-aba1-8c6aadbf9bba admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:56.110525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0c734-8e86-4f4f-aba1-8c6aadbf9bba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:06:56.110834 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0c734-8e86-4f4f-aba1-8c6aadbf9bba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:56.112471 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0c734-8e86-4f4f-aba1-8c6aadbf9bba admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:56.134216 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 766/1542] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:06:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:06:56.142620 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b85a0-c034-4bab-9716-c2f279fdeb7a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:56.156839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b85a0-c034-4bab-9716-c2f279fdeb7a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:56.164905 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e91658-b837-4d65-87ec-6e37cdbf59a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:56.165572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e91658-b837-4d65-87ec-6e37cdbf59a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:56.165853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e91658-b837-4d65-87ec-6e37cdbf59a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:56.166126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e91658-b837-4d65-87ec-6e37cdbf59a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:56.166823 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e91658-b837-4d65-87ec-6e37cdbf59a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:56.168768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e91658-b837-4d65-87ec-6e37cdbf59a6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:56.292487 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 777/1543] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:06:56 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:06:56.296617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0c734-8e86-4f4f-aba1-8c6aadbf9bba admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:56.309536 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0c734-8e86-4f4f-aba1-8c6aadbf9bba admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:56.361653 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d789b9-73dc-44ba-959e-757f44188267 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:56.361998 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d789b9-73dc-44ba-959e-757f44188267 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:56.362211 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d789b9-73dc-44ba-959e-757f44188267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:56.388428 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d9149-2d75-413b-b833-aa2264cbab8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:56.389113 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d9149-2d75-413b-b833-aa2264cbab8c admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:56.389394 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d9149-2d75-413b-b833-aa2264cbab8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:56.389664 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d9149-2d75-413b-b833-aa2264cbab8c admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:56.390361 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d9149-2d75-413b-b833-aa2264cbab8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:56.392802 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d9149-2d75-413b-b833-aa2264cbab8c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:56.403923 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-21d789b9-73dc-44ba-959e-757f44188267 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:06:56.406196 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-21d789b9-73dc-44ba-959e-757f44188267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:06:56.421336 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 778/1544] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:06:56.438776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1d608-8532-457f-b436-5561be0ea026 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:56.439131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1d608-8532-457f-b436-5561be0ea026 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:56.439337 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde1d608-8532-457f-b436-5561be0ea026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:56.474851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-bde1d608-8532-457f-b436-5561be0ea026 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:06:56.476068 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bde1d608-8532-457f-b436-5561be0ea026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:06:56.490327 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 779/1545] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:06:56.497475 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1d608-8532-457f-b436-5561be0ea026 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:56.512914 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde1d608-8532-457f-b436-5561be0ea026 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:56.521932 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a2600e-bec7-4e18-8d53-5515af37c276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:56.522617 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2600e-bec7-4e18-8d53-5515af37c276 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:56.522898 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2600e-bec7-4e18-8d53-5515af37c276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:56.523170 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2600e-bec7-4e18-8d53-5515af37c276 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:56.523913 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2600e-bec7-4e18-8d53-5515af37c276 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:06:56.524220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2600e-bec7-4e18-8d53-5515af37c276 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:56.525335 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: WARNING py.warnings [None req-96a2600e-bec7-4e18-8d53-5515af37c276 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 20:06:56.525335 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: warnings.warn(msg) Jun 23 20:06:56.525335 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: Jun 23 20:06:56.526259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2600e-bec7-4e18-8d53-5515af37c276 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:56.589428 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 780/1546] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:06:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:06:56.589428 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a2600e-bec7-4e18-8d53-5515af37c276 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:56.589428 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a2600e-bec7-4e18-8d53-5515af37c276 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:56.589428 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b95696-7245-4c0c-b3ee-772bf352b31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:56.589905 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b95696-7245-4c0c-b3ee-772bf352b31a admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:56.589905 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b95696-7245-4c0c-b3ee-772bf352b31a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:56.589905 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b95696-7245-4c0c-b3ee-772bf352b31a admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:56.589905 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b95696-7245-4c0c-b3ee-772bf352b31a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:56.589905 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b95696-7245-4c0c-b3ee-772bf352b31a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:56.742616 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 767/1547] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:06:56 2022] POST /identity/v3/users => generated 287 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:06:56.744205 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f40d71-43c4-4361-a278-9c1542772a2b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:56.744516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f40d71-43c4-4361-a278-9c1542772a2b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:56.744844 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f40d71-43c4-4361-a278-9c1542772a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:56.746149 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 781/1548] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:06:56 2022] POST /identity/v3/projects => generated 385 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:06:56.747616 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15057ff-0311-4bc8-a5f8-4a937e27fd69 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:56.747906 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15057ff-0311-4bc8-a5f8-4a937e27fd69 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:56.748182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15057ff-0311-4bc8-a5f8-4a937e27fd69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:56.773909 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-27f40d71-43c4-4361-a278-9c1542772a2b None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:06:56.775343 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-27f40d71-43c4-4361-a278-9c1542772a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:06:56.784141 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f15057ff-0311-4bc8-a5f8-4a937e27fd69 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:06:56.809292 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f15057ff-0311-4bc8-a5f8-4a937e27fd69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:06:56.886683 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 768/1549] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:06:56.887195 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f40d71-43c4-4361-a278-9c1542772a2b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:56.906800 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f40d71-43c4-4361-a278-9c1542772a2b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:56.914118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4923ee-37ad-403b-9d93-c7a10a4d52c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:56.914814 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 782/1550] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:06:56.915009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4923ee-37ad-403b-9d93-c7a10a4d52c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:56.915287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4923ee-37ad-403b-9d93-c7a10a4d52c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:56.915406 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15057ff-0311-4bc8-a5f8-4a937e27fd69 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:56.915578 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4923ee-37ad-403b-9d93-c7a10a4d52c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:56.916347 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4923ee-37ad-403b-9d93-c7a10a4d52c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:56.918011 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4923ee-37ad-403b-9d93-c7a10a4d52c8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:56.928544 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15057ff-0311-4bc8-a5f8-4a937e27fd69 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:56.932338 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4906ba-f641-41e9-8716-11f9d60746e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:56.933035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4906ba-f641-41e9-8716-11f9d60746e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:56.933323 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4906ba-f641-41e9-8716-11f9d60746e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:56.933596 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4906ba-f641-41e9-8716-11f9d60746e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:56.934308 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4906ba-f641-41e9-8716-11f9d60746e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:56.935358 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: WARNING py.warnings [None req-fe4906ba-f641-41e9-8716-11f9d60746e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 20:06:56.935358 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: warnings.warn(msg) Jun 23 20:06:56.935358 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: Jun 23 20:06:56.937926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 769/1551] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:06:56.939509 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b551c5-abee-4877-be56-60cb18e965d2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:56.940040 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b551c5-abee-4877-be56-60cb18e965d2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:56.940704 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b551c5-abee-4877-be56-60cb18e965d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:56.945111 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4906ba-f641-41e9-8716-11f9d60746e5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:56.965442 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e4b551c5-abee-4877-be56-60cb18e965d2 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:06:56.967621 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e4b551c5-abee-4877-be56-60cb18e965d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:06:56.974487 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 770/1552] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:06:56.975792 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784bc390-4985-415e-8b99-ca715241201e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:56.976082 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784bc390-4985-415e-8b99-ca715241201e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:56.976439 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784bc390-4985-415e-8b99-ca715241201e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:56.999278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-784bc390-4985-415e-8b99-ca715241201e None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:06:57.010941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-784bc390-4985-415e-8b99-ca715241201e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:06:57.010941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 771/1553] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:06:57.010941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c88ef-cc6c-4e60-a88d-a3b2663f8f48 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:57.010941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c88ef-cc6c-4e60-a88d-a3b2663f8f48 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:57.011309 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c88ef-cc6c-4e60-a88d-a3b2663f8f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:57.029821 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-991c88ef-cc6c-4e60-a88d-a3b2663f8f48 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:06:57.031343 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-991c88ef-cc6c-4e60-a88d-a3b2663f8f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:06:57.039346 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 772/1554] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:06:57.040647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02c506-f1b3-4539-9191-2086f82b3af0 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:57.040937 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02c506-f1b3-4539-9191-2086f82b3af0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:57.041215 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02c506-f1b3-4539-9191-2086f82b3af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:57.060004 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7f02c506-f1b3-4539-9191-2086f82b3af0 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:06:57.061247 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7f02c506-f1b3-4539-9191-2086f82b3af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:06:57.067925 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 783/1555] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:06:56 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:06:57.067925 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4906ba-f641-41e9-8716-11f9d60746e5 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:57.073388 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 773/1556] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:06:57.074626 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ea0bc-8495-4111-986f-2aa7d4113140 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:57.074917 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ea0bc-8495-4111-986f-2aa7d4113140 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:57.075194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ea0bc-8495-4111-986f-2aa7d4113140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:57.078396 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4906ba-f641-41e9-8716-11f9d60746e5 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:57.083097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5779ac-173c-4cfc-b017-f75c3ba9677e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:57.083749 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5779ac-173c-4cfc-b017-f75c3ba9677e admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:57.084031 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5779ac-173c-4cfc-b017-f75c3ba9677e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:57.084326 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5779ac-173c-4cfc-b017-f75c3ba9677e admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:57.085057 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5779ac-173c-4cfc-b017-f75c3ba9677e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:57.086775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5779ac-173c-4cfc-b017-f75c3ba9677e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:57.093367 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8d1ea0bc-8495-4111-986f-2aa7d4113140 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:06:57.094564 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8d1ea0bc-8495-4111-986f-2aa7d4113140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:06:57.097559 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 784/1557] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:06:57.098841 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e6e2c-295c-4bdf-826c-2457c13aad6f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:57.099131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e6e2c-295c-4bdf-826c-2457c13aad6f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:57.099407 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e6e2c-295c-4bdf-826c-2457c13aad6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:57.104170 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 774/1558] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:06:57.104971 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ea0bc-8495-4111-986f-2aa7d4113140 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:57.114929 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ea0bc-8495-4111-986f-2aa7d4113140 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:57.119432 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfc61b-45a1-4641-a968-42382921fe68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:57.120083 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfc61b-45a1-4641-a968-42382921fe68 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:57.120395 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfc61b-45a1-4641-a968-42382921fe68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:57.120673 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfc61b-45a1-4641-a968-42382921fe68 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:57.121397 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dfc61b-45a1-4641-a968-42382921fe68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:06:57.121706 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dfc61b-45a1-4641-a968-42382921fe68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:57.123304 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dfc61b-45a1-4641-a968-42382921fe68 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:57.124828 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-677e6e2c-295c-4bdf-826c-2457c13aad6f None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:06:57.126029 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-677e6e2c-295c-4bdf-826c-2457c13aad6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:06:57.134398 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 785/1559] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:06:57.135678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1ef43-5854-432d-b5b1-fff5cb429769 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:57.135975 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1ef43-5854-432d-b5b1-fff5cb429769 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:57.136094 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 775/1560] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:06:57.136290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1ef43-5854-432d-b5b1-fff5cb429769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:57.137409 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62edb7-9016-452a-a30e-8b1bc252c227 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:57.137638 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62edb7-9016-452a-a30e-8b1bc252c227 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:57.137923 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62edb7-9016-452a-a30e-8b1bc252c227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:57.160082 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6f62edb7-9016-452a-a30e-8b1bc252c227 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:06:57.160982 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-01b1ef43-5854-432d-b5b1-fff5cb429769 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:06:57.161300 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6f62edb7-9016-452a-a30e-8b1bc252c227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:06:57.162154 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-01b1ef43-5854-432d-b5b1-fff5cb429769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:06:57.169156 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 776/1561] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:06:57.173185 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62edb7-9016-452a-a30e-8b1bc252c227 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:57.173303 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 786/1562] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:06:57.173303 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b1ef43-5854-432d-b5b1-fff5cb429769 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:57.181062 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b1ef43-5854-432d-b5b1-fff5cb429769 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:57.183923 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62edb7-9016-452a-a30e-8b1bc252c227 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:57.185691 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1530c-c9ed-4c68-b6b3-b97273ccbb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:57.186397 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1530c-c9ed-4c68-b6b3-b97273ccbb71 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:57.186754 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1530c-c9ed-4c68-b6b3-b97273ccbb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:57.186940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1530c-c9ed-4c68-b6b3-b97273ccbb71 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:57.187725 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1530c-c9ed-4c68-b6b3-b97273ccbb71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:06:57.188034 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1530c-c9ed-4c68-b6b3-b97273ccbb71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:57.188798 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c807a-4498-4960-a4d5-d54332022cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:57.189441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c807a-4498-4960-a4d5-d54332022cac admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:57.189683 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1530c-c9ed-4c68-b6b3-b97273ccbb71 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:57.189789 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c807a-4498-4960-a4d5-d54332022cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:57.189977 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c807a-4498-4960-a4d5-d54332022cac admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:57.190723 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652c807a-4498-4960-a4d5-d54332022cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:57.192936 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652c807a-4498-4960-a4d5-d54332022cac admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:57.194447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:06:57.202064 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 787/1563] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:06:57.208715 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1530c-c9ed-4c68-b6b3-b97273ccbb71 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:57.208862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 777/1564] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:06:57.208862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cfff4-7b05-467a-9d70-a47e2c356a18 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:57.208862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cfff4-7b05-467a-9d70-a47e2c356a18 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:57.208862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cfff4-7b05-467a-9d70-a47e2c356a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:57.213678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1530c-c9ed-4c68-b6b3-b97273ccbb71 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:57.218399 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cfb7c9-d687-492f-8b52-fe1655e2c983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:57.219247 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfb7c9-d687-492f-8b52-fe1655e2c983 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:57.219548 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfb7c9-d687-492f-8b52-fe1655e2c983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:57.219805 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfb7c9-d687-492f-8b52-fe1655e2c983 admin admin] PATH_INFO: `/v3/projects/ebbc37447b7a44a88daf3f4da9466a64/users/33465d9bb2424ae196ed04919b76ed46/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:57.225956 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-116cfff4-7b05-467a-9d70-a47e2c356a18 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:06:57.227148 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-116cfff4-7b05-467a-9d70-a47e2c356a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:06:57.235263 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 778/1565] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:06:57.236111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116cfff4-7b05-467a-9d70-a47e2c356a18 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:57.243477 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cfb7c9-d687-492f-8b52-fe1655e2c983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebbc37447b7a44a88daf3f4da9466a64, user_id=33465d9bb2424ae196ed04919b76ed46, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:57.245931 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116cfff4-7b05-467a-9d70-a47e2c356a18 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:57.246244 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cfb7c9-d687-492f-8b52-fe1655e2c983 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:57.250378 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6d8fe-757b-4763-99cb-69f4c45e191b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:57.251008 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6d8fe-757b-4763-99cb-69f4c45e191b admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:57.251285 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6d8fe-757b-4763-99cb-69f4c45e191b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:57.251554 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6d8fe-757b-4763-99cb-69f4c45e191b admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:57.252275 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f6d8fe-757b-4763-99cb-69f4c45e191b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:57.254545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: WARNING py.warnings [None req-c0f6d8fe-757b-4763-99cb-69f4c45e191b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 20:06:57.254545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: warnings.warn(msg) Jun 23 20:06:57.254545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: Jun 23 20:06:57.255750 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f6d8fe-757b-4763-99cb-69f4c45e191b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:57.339327 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 788/1566] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:06:57 2022] PUT /identity/v3/projects/ebbc37447b7a44a88daf3f4da9466a64/users/33465d9bb2424ae196ed04919b76ed46/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:06:57.340824 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cfb7c9-d687-492f-8b52-fe1655e2c983 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:57.346187 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:06:57.355534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 779/1567] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:06:57 2022] POST /identity/v3/projects => generated 391 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:06:57.355534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6d8fe-757b-4763-99cb-69f4c45e191b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:57.405251 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cfb7c9-d687-492f-8b52-fe1655e2c983 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:57.415569 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6d8fe-757b-4763-99cb-69f4c45e191b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:57.420257 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb9e10-44ae-4f41-972c-15c08e1e2b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:57.420937 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb9e10-44ae-4f41-972c-15c08e1e2b26 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:57.421219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb9e10-44ae-4f41-972c-15c08e1e2b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:57.421488 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb9e10-44ae-4f41-972c-15c08e1e2b26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:57.422204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb9e10-44ae-4f41-972c-15c08e1e2b26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:06:57.422542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb9e10-44ae-4f41-972c-15c08e1e2b26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:57.424149 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb9e10-44ae-4f41-972c-15c08e1e2b26 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:57.436570 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 780/1568] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:06:57.441875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23449adf-9092-4e7f-b154-af589590caca None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:57.442285 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23449adf-9092-4e7f-b154-af589590caca None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:57.442574 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23449adf-9092-4e7f-b154-af589590caca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:57.460923 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-23449adf-9092-4e7f-b154-af589590caca None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:06:57.462124 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-23449adf-9092-4e7f-b154-af589590caca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:06:57.463077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98911555-6b7c-4a26-abc5-526e2174b65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:57.463855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98911555-6b7c-4a26-abc5-526e2174b65e admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:57.464114 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98911555-6b7c-4a26-abc5-526e2174b65e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:57.464432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98911555-6b7c-4a26-abc5-526e2174b65e admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:57.465161 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98911555-6b7c-4a26-abc5-526e2174b65e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:06:57.465471 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98911555-6b7c-4a26-abc5-526e2174b65e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:57.467073 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98911555-6b7c-4a26-abc5-526e2174b65e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:57.470601 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 781/1569] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:06:57.471391 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23449adf-9092-4e7f-b154-af589590caca None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:57.480213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 789/1570] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:06:57.481043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23449adf-9092-4e7f-b154-af589590caca None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:57.481400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98911555-6b7c-4a26-abc5-526e2174b65e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:57.485714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee405a-70c6-4a6d-b84c-14c9aac4634f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:57.486363 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee405a-70c6-4a6d-b84c-14c9aac4634f admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:57.486647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee405a-70c6-4a6d-b84c-14c9aac4634f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:57.486922 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee405a-70c6-4a6d-b84c-14c9aac4634f admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:57.487647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee405a-70c6-4a6d-b84c-14c9aac4634f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:57.489851 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee405a-70c6-4a6d-b84c-14c9aac4634f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:57.492679 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98911555-6b7c-4a26-abc5-526e2174b65e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:57.497536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c7142-c0de-455d-884d-bd94eceda35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:57.498172 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c7142-c0de-455d-884d-bd94eceda35b admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:57.498464 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c7142-c0de-455d-884d-bd94eceda35b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:57.498738 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c7142-c0de-455d-884d-bd94eceda35b admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:57.499418 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c7142-c0de-455d-884d-bd94eceda35b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:57.500014 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 782/1571] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:06:57.500834 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee405a-70c6-4a6d-b84c-14c9aac4634f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:57.504412 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c7142-c0de-455d-884d-bd94eceda35b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:57.510640 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee405a-70c6-4a6d-b84c-14c9aac4634f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:57.515528 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e690e46-66e0-4f3a-beda-cda8c6d49101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:57.516168 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e690e46-66e0-4f3a-beda-cda8c6d49101 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:57.516485 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e690e46-66e0-4f3a-beda-cda8c6d49101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:57.516762 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e690e46-66e0-4f3a-beda-cda8c6d49101 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:57.517508 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e690e46-66e0-4f3a-beda-cda8c6d49101 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:06:57.517818 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e690e46-66e0-4f3a-beda-cda8c6d49101 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:57.519452 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e690e46-66e0-4f3a-beda-cda8c6d49101 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:57.531387 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 783/1572] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:06:57.532620 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1d6df-3bc5-4cca-893f-7848443b737e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:57.533564 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1d6df-3bc5-4cca-893f-7848443b737e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:57.533564 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1d6df-3bc5-4cca-893f-7848443b737e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:57.559042 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-48a1d6df-3bc5-4cca-893f-7848443b737e None None] MFA Rules not processed for user `33465d9bb2424ae196ed04919b76ed46`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:06:57.560250 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-48a1d6df-3bc5-4cca-893f-7848443b737e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:06:57.570375 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 790/1573] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:06:57 2022] POST /identity/v3/projects => generated 363 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:06:57.571220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c7142-c0de-455d-884d-bd94eceda35b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:57.626647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 784/1574] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:06:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:06:57.627536 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1d6df-3bc5-4cca-893f-7848443b737e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:57.633047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c7142-c0de-455d-884d-bd94eceda35b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:57.637649 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e1cfd-31b1-4e2d-a4a4-91f67ad511b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:57.638320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e1cfd-31b1-4e2d-a4a4-91f67ad511b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:57.638602 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e1cfd-31b1-4e2d-a4a4-91f67ad511b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:57.638875 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e1cfd-31b1-4e2d-a4a4-91f67ad511b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:57.639568 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e1cfd-31b1-4e2d-a4a4-91f67ad511b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:57.641530 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e1cfd-31b1-4e2d-a4a4-91f67ad511b3 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:57.642358 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1d6df-3bc5-4cca-893f-7848443b737e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:57.649869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054de459-7098-4c86-8a81-da128d825cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:57.650530 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054de459-7098-4c86-8a81-da128d825cd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:57.650811 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054de459-7098-4c86-8a81-da128d825cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:57.651086 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054de459-7098-4c86-8a81-da128d825cd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:57.651778 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054de459-7098-4c86-8a81-da128d825cd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:57.653720 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054de459-7098-4c86-8a81-da128d825cd7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:57.803144 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 791/1575] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:06:57 2022] POST /identity/v3/users => generated 296 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:06:57.803898 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e1cfd-31b1-4e2d-a4a4-91f67ad511b3 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:57.813472 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e1cfd-31b1-4e2d-a4a4-91f67ad511b3 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:57.818038 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02361382-47f1-4c11-96b8-ec261ba67a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:57.818657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02361382-47f1-4c11-96b8-ec261ba67a6b admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:57.818941 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02361382-47f1-4c11-96b8-ec261ba67a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:57.819216 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02361382-47f1-4c11-96b8-ec261ba67a6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:57.819919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02361382-47f1-4c11-96b8-ec261ba67a6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:06:57.820225 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02361382-47f1-4c11-96b8-ec261ba67a6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:57.822102 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02361382-47f1-4c11-96b8-ec261ba67a6b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:57.830513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:06:57.834668 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 792/1576] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:06:57.835432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02361382-47f1-4c11-96b8-ec261ba67a6b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:57.837656 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:06:57.845289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 785/1577] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:06:57 2022] POST /identity/v3/projects => generated 363 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:06:57.846568 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02361382-47f1-4c11-96b8-ec261ba67a6b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:57.846698 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054de459-7098-4c86-8a81-da128d825cd7 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:57.904204 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2f4cd-9cef-4a30-947c-d69dec724d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:57.904899 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2f4cd-9cef-4a30-947c-d69dec724d8b admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:57.905183 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2f4cd-9cef-4a30-947c-d69dec724d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:57.905455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2f4cd-9cef-4a30-947c-d69dec724d8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:57.906148 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2f4cd-9cef-4a30-947c-d69dec724d8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:57.907996 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054de459-7098-4c86-8a81-da128d825cd7 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:57.908095 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2f4cd-9cef-4a30-947c-d69dec724d8b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:57.914397 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc88f4-0d6b-4698-9aee-e0af2e552e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:57.915120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc88f4-0d6b-4698-9aee-e0af2e552e0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:57.915120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc88f4-0d6b-4698-9aee-e0af2e552e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:57.915502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc88f4-0d6b-4698-9aee-e0af2e552e0d admin admin] PATH_INFO: `/v3/projects/bd810755b80346ae9219a2d0c61a8a06/users/8a5b19c8c0d44e1f93bb1dea72884cc3/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:57.930068 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc88f4-0d6b-4698-9aee-e0af2e552e0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd810755b80346ae9219a2d0c61a8a06, user_id=8a5b19c8c0d44e1f93bb1dea72884cc3, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:57.931120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: WARNING py.warnings [None req-74fc88f4-0d6b-4698-9aee-e0af2e552e0d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 20:06:57.931120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: warnings.warn(msg) Jun 23 20:06:57.931120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: Jun 23 20:06:57.933111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc88f4-0d6b-4698-9aee-e0af2e552e0d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:58.170173 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 793/1578] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:06:57 2022] POST /identity/v3/projects => generated 379 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:06:58.171627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 786/1579] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:06:57 2022] PUT /identity/v3/projects/bd810755b80346ae9219a2d0c61a8a06/users/8a5b19c8c0d44e1f93bb1dea72884cc3/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:06:58.172076 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2f4cd-9cef-4a30-947c-d69dec724d8b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:58.172659 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc88f4-0d6b-4698-9aee-e0af2e552e0d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:58.252794 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc88f4-0d6b-4698-9aee-e0af2e552e0d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:58.260599 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06717a46-891e-401a-97d4-ff70ba94abb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:58.261303 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2f4cd-9cef-4a30-947c-d69dec724d8b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:58.261452 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06717a46-891e-401a-97d4-ff70ba94abb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:58.261691 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06717a46-891e-401a-97d4-ff70ba94abb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:58.262093 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06717a46-891e-401a-97d4-ff70ba94abb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:58.262853 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06717a46-891e-401a-97d4-ff70ba94abb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:58.265108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06717a46-891e-401a-97d4-ff70ba94abb7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:58.271375 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1d98a-d85e-47c3-8b51-fb729b18248e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:58.271375 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1d98a-d85e-47c3-8b51-fb729b18248e admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:58.271375 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1d98a-d85e-47c3-8b51-fb729b18248e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:58.271375 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1d98a-d85e-47c3-8b51-fb729b18248e admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:58.272014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1d98a-d85e-47c3-8b51-fb729b18248e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:58.272014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1d98a-d85e-47c3-8b51-fb729b18248e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:58.368786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 787/1580] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:06:58 2022] POST /identity/v3/users => generated 282 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:06:58.369616 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06717a46-891e-401a-97d4-ff70ba94abb7 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:58.380350 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06717a46-891e-401a-97d4-ff70ba94abb7 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:58.384942 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 794/1581] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:06:58 2022] POST /identity/v3/projects => generated 385 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:06:58.385221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a581c-2182-4edc-b91e-c570b195e91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:58.385830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1d98a-d85e-47c3-8b51-fb729b18248e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:58.385941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a581c-2182-4edc-b91e-c570b195e91c admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:58.386162 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a581c-2182-4edc-b91e-c570b195e91c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:58.386445 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a581c-2182-4edc-b91e-c570b195e91c admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:58.387140 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15a581c-2182-4edc-b91e-c570b195e91c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:58.392371 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15a581c-2182-4edc-b91e-c570b195e91c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:58.456814 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1d98a-d85e-47c3-8b51-fb729b18248e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:58.463482 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6c724-4be4-4229-ab5b-5e12ff721454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:58.467244 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6c724-4be4-4229-ab5b-5e12ff721454 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:58.467244 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6c724-4be4-4229-ab5b-5e12ff721454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:58.467244 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6c724-4be4-4229-ab5b-5e12ff721454 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:58.467244 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6c724-4be4-4229-ab5b-5e12ff721454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:58.467244 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6c724-4be4-4229-ab5b-5e12ff721454 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:58.477237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 795/1582] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:06:58.478274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6c724-4be4-4229-ab5b-5e12ff721454 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:58.492832 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6c724-4be4-4229-ab5b-5e12ff721454 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:58.498009 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfffc3-fb60-48cf-9266-5b7dea922840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:58.502017 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfffc3-fb60-48cf-9266-5b7dea922840 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:58.502017 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfffc3-fb60-48cf-9266-5b7dea922840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:58.502017 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfffc3-fb60-48cf-9266-5b7dea922840 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:58.502017 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bfffc3-fb60-48cf-9266-5b7dea922840 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:58.502431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 788/1583] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:06:58 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:06:58.502431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a581c-2182-4edc-b91e-c570b195e91c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:58.502646 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bfffc3-fb60-48cf-9266-5b7dea922840 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:58.507099 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:06:58.554416 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:06:58.568277 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a581c-2182-4edc-b91e-c570b195e91c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:58.574031 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c1160-3005-4fe9-b513-d094e32557b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:58.574741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c1160-3005-4fe9-b513-d094e32557b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:58.575024 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c1160-3005-4fe9-b513-d094e32557b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:58.575297 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c1160-3005-4fe9-b513-d094e32557b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:58.575995 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c1160-3005-4fe9-b513-d094e32557b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:58.578061 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c1160-3005-4fe9-b513-d094e32557b7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:58.599260 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 796/1584] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:06:58 2022] POST /identity/v3/projects => generated 389 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:06:58.600194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfffc3-fb60-48cf-9266-5b7dea922840 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:58.667503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bfffc3-fb60-48cf-9266-5b7dea922840 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:58.672126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b4fa4c-e78f-4570-abdb-9c3d69cb1170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:58.672866 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4fa4c-e78f-4570-abdb-9c3d69cb1170 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:58.673161 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4fa4c-e78f-4570-abdb-9c3d69cb1170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:58.673434 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4fa4c-e78f-4570-abdb-9c3d69cb1170 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:58.674124 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b4fa4c-e78f-4570-abdb-9c3d69cb1170 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:58.676722 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b4fa4c-e78f-4570-abdb-9c3d69cb1170 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:58.699962 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 789/1585] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:06:58 2022] POST /identity/v3/users => generated 282 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:06:58.701311 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887a0e5-ab8c-497e-8e17-b40ca0fc563b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:58.701598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887a0e5-ab8c-497e-8e17-b40ca0fc563b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:58.702153 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887a0e5-ab8c-497e-8e17-b40ca0fc563b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:58.719875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8887a0e5-ab8c-497e-8e17-b40ca0fc563b None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:06:58.722076 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8887a0e5-ab8c-497e-8e17-b40ca0fc563b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:06:58.729812 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 790/1586] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:06:58.730593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887a0e5-ab8c-497e-8e17-b40ca0fc563b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:58.742450 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887a0e5-ab8c-497e-8e17-b40ca0fc563b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:58.749153 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10731c2-2195-431a-b055-d94bbcab74d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:58.749731 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10731c2-2195-431a-b055-d94bbcab74d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:58.750011 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10731c2-2195-431a-b055-d94bbcab74d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:58.750284 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10731c2-2195-431a-b055-d94bbcab74d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:58.751010 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10731c2-2195-431a-b055-d94bbcab74d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:58.752717 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10731c2-2195-431a-b055-d94bbcab74d4 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:58.763425 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 791/1587] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:06:58.764205 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10731c2-2195-431a-b055-d94bbcab74d4 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:58.776336 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10731c2-2195-431a-b055-d94bbcab74d4 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:58.780841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75a734-1653-4998-8c51-209824bcbdd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:58.781478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75a734-1653-4998-8c51-209824bcbdd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:58.781760 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75a734-1653-4998-8c51-209824bcbdd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:58.782032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75a734-1653-4998-8c51-209824bcbdd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:58.782714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a75a734-1653-4998-8c51-209824bcbdd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:58.785151 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a75a734-1653-4998-8c51-209824bcbdd1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:58.796735 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 797/1588] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:06:58 2022] POST /identity/v3/users => generated 290 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:06:58.802852 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b4fa4c-e78f-4570-abdb-9c3d69cb1170 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:58.807967 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b4fa4c-e78f-4570-abdb-9c3d69cb1170 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:58.812484 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa3616-b0b4-4265-adf9-b1840ecc52db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:58.813125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa3616-b0b4-4265-adf9-b1840ecc52db admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:58.813635 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa3616-b0b4-4265-adf9-b1840ecc52db admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:58.813635 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa3616-b0b4-4265-adf9-b1840ecc52db admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:58.814372 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa3616-b0b4-4265-adf9-b1840ecc52db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:58.815974 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa3616-b0b4-4265-adf9-b1840ecc52db admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:58.826010 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 798/1589] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:06:58.826766 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa3616-b0b4-4265-adf9-b1840ecc52db admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:58.836469 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa3616-b0b4-4265-adf9-b1840ecc52db admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:58.841982 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76bbd01-ad79-44bc-afc7-0efb61168004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:58.842613 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bbd01-ad79-44bc-afc7-0efb61168004 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:58.842891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bbd01-ad79-44bc-afc7-0efb61168004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:58.843162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bbd01-ad79-44bc-afc7-0efb61168004 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:58.843833 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76bbd01-ad79-44bc-afc7-0efb61168004 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:58.845765 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76bbd01-ad79-44bc-afc7-0efb61168004 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:58.922850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 792/1590] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:06:58 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:06:58.923673 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75a734-1653-4998-8c51-209824bcbdd1 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:58.933829 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75a734-1653-4998-8c51-209824bcbdd1 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:58.938379 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b4b830-ea4a-42db-bf0a-8cc4456281ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:58.939047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4b830-ea4a-42db-bf0a-8cc4456281ea admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:58.939333 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4b830-ea4a-42db-bf0a-8cc4456281ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:58.939610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4b830-ea4a-42db-bf0a-8cc4456281ea admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:58.940287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b4b830-ea4a-42db-bf0a-8cc4456281ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:58.942298 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b4b830-ea4a-42db-bf0a-8cc4456281ea admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:58.973046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 799/1591] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:06:58 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:06:58.974026 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76bbd01-ad79-44bc-afc7-0efb61168004 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:58.983560 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76bbd01-ad79-44bc-afc7-0efb61168004 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:58.988701 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9f4c0-4710-476b-8577-b0adfef9a760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:58.989379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9f4c0-4710-476b-8577-b0adfef9a760 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:58.989657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9f4c0-4710-476b-8577-b0adfef9a760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:58.989926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9f4c0-4710-476b-8577-b0adfef9a760 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:58.990636 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c9f4c0-4710-476b-8577-b0adfef9a760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:58.992243 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c9f4c0-4710-476b-8577-b0adfef9a760 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:59.004084 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 800/1592] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:06:59.007119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9f4c0-4710-476b-8577-b0adfef9a760 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:59.056822 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9f4c0-4710-476b-8577-b0adfef9a760 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:59.060538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd592c3b-6055-4a33-8b49-7150b76a496a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:59.061197 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd592c3b-6055-4a33-8b49-7150b76a496a service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:59.061480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd592c3b-6055-4a33-8b49-7150b76a496a service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:59.061751 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd592c3b-6055-4a33-8b49-7150b76a496a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:59.066885 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 793/1593] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:06:58 2022] POST /identity/v3/users => generated 295 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:06:59.066885 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b4b830-ea4a-42db-bf0a-8cc4456281ea admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:59.074484 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd592c3b-6055-4a33-8b49-7150b76a496a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:59.075953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd592c3b-6055-4a33-8b49-7150b76a496a service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:59.076472 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b4b830-ea4a-42db-bf0a-8cc4456281ea admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:59.081747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0696965-c467-4171-9b52-eb0a198c6310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:59.082426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0696965-c467-4171-9b52-eb0a198c6310 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:59.082706 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0696965-c467-4171-9b52-eb0a198c6310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:59.082980 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0696965-c467-4171-9b52-eb0a198c6310 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:59.083703 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0696965-c467-4171-9b52-eb0a198c6310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:59.085396 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0696965-c467-4171-9b52-eb0a198c6310 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:59.085581 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 801/1594] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:06:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:06:59.086432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd592c3b-6055-4a33-8b49-7150b76a496a service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:59.095881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd592c3b-6055-4a33-8b49-7150b76a496a service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:59.102070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 794/1595] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:06:59.102070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0696965-c467-4171-9b52-eb0a198c6310 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:59.104714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa5205b-88c6-4569-8056-54007cd15d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:59.105279 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5205b-88c6-4569-8056-54007cd15d8c admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:59.105559 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5205b-88c6-4569-8056-54007cd15d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:59.105834 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa5205b-88c6-4569-8056-54007cd15d8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:59.106813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa5205b-88c6-4569-8056-54007cd15d8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:59.108284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa5205b-88c6-4569-8056-54007cd15d8c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:59.112651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0696965-c467-4171-9b52-eb0a198c6310 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:59.118091 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00598a7b-e7b0-4373-bc00-95646780df24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:59.119072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00598a7b-e7b0-4373-bc00-95646780df24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:59.119222 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 802/1596] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:06:59.120156 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa5205b-88c6-4569-8056-54007cd15d8c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:59.122307 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00598a7b-e7b0-4373-bc00-95646780df24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:59.122619 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00598a7b-e7b0-4373-bc00-95646780df24 admin admin] PATH_INFO: `/v3/projects/bf37c9d0240b48078038d70f678a93eb/users/f041449191c94a5ca5a38c29dd146a15/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:59.129701 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa5205b-88c6-4569-8056-54007cd15d8c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:59.134330 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43decbc-f849-42c4-a318-3083a888846c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:59.134961 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43decbc-f849-42c4-a318-3083a888846c admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:59.135242 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43decbc-f849-42c4-a318-3083a888846c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:59.135515 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43decbc-f849-42c4-a318-3083a888846c admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:59.136225 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43decbc-f849-42c4-a318-3083a888846c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:59.137629 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00598a7b-e7b0-4373-bc00-95646780df24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf37c9d0240b48078038d70f678a93eb, user_id=f041449191c94a5ca5a38c29dd146a15, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:59.138005 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43decbc-f849-42c4-a318-3083a888846c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:59.140262 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00598a7b-e7b0-4373-bc00-95646780df24 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:59.148630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 803/1597] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:06:59.149481 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43decbc-f849-42c4-a318-3083a888846c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:59.159059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43decbc-f849-42c4-a318-3083a888846c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:59.164467 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03962ef-bc37-4811-8451-745932c8bffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:59.165122 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03962ef-bc37-4811-8451-745932c8bffc admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:59.165433 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03962ef-bc37-4811-8451-745932c8bffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:59.165709 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03962ef-bc37-4811-8451-745932c8bffc admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:59.166414 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03962ef-bc37-4811-8451-745932c8bffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:59.168031 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03962ef-bc37-4811-8451-745932c8bffc admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:59.178334 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 804/1598] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:06:59.185891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03962ef-bc37-4811-8451-745932c8bffc admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:59.185891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:06:59.188517 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03962ef-bc37-4811-8451-745932c8bffc admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:59.193080 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67bc3da-a8aa-4de5-ab15-ece04c947082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:59.193709 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67bc3da-a8aa-4de5-ab15-ece04c947082 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:59.193989 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67bc3da-a8aa-4de5-ab15-ece04c947082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:59.194259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67bc3da-a8aa-4de5-ab15-ece04c947082 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:59.194983 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67bc3da-a8aa-4de5-ab15-ece04c947082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:59.196858 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67bc3da-a8aa-4de5-ab15-ece04c947082 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:59.208551 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 805/1599] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:06:59.208551 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67bc3da-a8aa-4de5-ab15-ece04c947082 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:59.211728 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 795/1600] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:06:59 2022] PUT /identity/v3/projects/bf37c9d0240b48078038d70f678a93eb/users/f041449191c94a5ca5a38c29dd146a15/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:06:59.214126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00598a7b-e7b0-4373-bc00-95646780df24 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:59.218155 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67bc3da-a8aa-4de5-ab15-ece04c947082 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:59.279688 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00598a7b-e7b0-4373-bc00-95646780df24 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:59.284420 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ad45d-a405-4028-b213-c0f2306df494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:59.285958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ad45d-a405-4028-b213-c0f2306df494 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:59.289631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ad45d-a405-4028-b213-c0f2306df494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:59.289797 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e635d-4265-4ffd-845e-51a9263a9ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:59.289953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ad45d-a405-4028-b213-c0f2306df494 admin admin] PATH_INFO: `/v3/projects/e9733cc852d54b15a7fab334a1195114/users/bb1d18a000dd48d1b50569857c1783c4/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:59.290060 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e635d-4265-4ffd-845e-51a9263a9ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:59.290060 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e635d-4265-4ffd-845e-51a9263a9ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:59.290060 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e635d-4265-4ffd-845e-51a9263a9ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:59.290060 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e635d-4265-4ffd-845e-51a9263a9ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:59.290060 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e635d-4265-4ffd-845e-51a9263a9ff0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:59.302346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 806/1601] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:06:59.302346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e635d-4265-4ffd-845e-51a9263a9ff0 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:59.302625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ad45d-a405-4028-b213-c0f2306df494 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9733cc852d54b15a7fab334a1195114, user_id=bb1d18a000dd48d1b50569857c1783c4, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:59.304096 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ad45d-a405-4028-b213-c0f2306df494 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:59.312486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e635d-4265-4ffd-845e-51a9263a9ff0 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:59.317113 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebbb0e2-a860-49e6-ac8a-e1dd98451081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:59.318513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbb0e2-a860-49e6-ac8a-e1dd98451081 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:59.318513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbb0e2-a860-49e6-ac8a-e1dd98451081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:59.318513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbb0e2-a860-49e6-ac8a-e1dd98451081 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:59.318964 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebbb0e2-a860-49e6-ac8a-e1dd98451081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:59.320566 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebbb0e2-a860-49e6-ac8a-e1dd98451081 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:59.330337 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 807/1602] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:06:59.331222 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebbb0e2-a860-49e6-ac8a-e1dd98451081 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:59.340861 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebbb0e2-a860-49e6-ac8a-e1dd98451081 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:59.345919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3420e-56b4-4ccf-9140-807c53f2a46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:59.346544 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3420e-56b4-4ccf-9140-807c53f2a46f admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:59.346821 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3420e-56b4-4ccf-9140-807c53f2a46f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:59.347139 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3420e-56b4-4ccf-9140-807c53f2a46f admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:59.348429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3420e-56b4-4ccf-9140-807c53f2a46f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:59.350287 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3420e-56b4-4ccf-9140-807c53f2a46f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:59.361671 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 808/1603] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:06:59.362593 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3420e-56b4-4ccf-9140-807c53f2a46f admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:59.374167 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3420e-56b4-4ccf-9140-807c53f2a46f admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:59.379189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b0590e-4c7e-4db4-9ae4-0a901b738f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:59.380072 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0590e-4c7e-4db4-9ae4-0a901b738f19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:59.380363 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0590e-4c7e-4db4-9ae4-0a901b738f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:59.380627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0590e-4c7e-4db4-9ae4-0a901b738f19 admin admin] PATH_INFO: `/v3/projects/dbb3b7f14b6647e1b250dba5d6aead39/users/2ed0f50aa4604d74ae75571524acfeb4/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:59.388228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 796/1604] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:06:59 2022] PUT /identity/v3/projects/e9733cc852d54b15a7fab334a1195114/users/bb1d18a000dd48d1b50569857c1783c4/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:06:59.388228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ad45d-a405-4028-b213-c0f2306df494 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:59.401635 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b0590e-4c7e-4db4-9ae4-0a901b738f19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbb3b7f14b6647e1b250dba5d6aead39, user_id=2ed0f50aa4604d74ae75571524acfeb4, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:59.406023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b0590e-4c7e-4db4-9ae4-0a901b738f19 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:59.432763 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:06:59.460555 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ad45d-a405-4028-b213-c0f2306df494 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:59.465324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c66cd-a71c-4a39-9db3-10f7e0475e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:59.466238 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c66cd-a71c-4a39-9db3-10f7e0475e2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:59.466521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c66cd-a71c-4a39-9db3-10f7e0475e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:59.466792 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c66cd-a71c-4a39-9db3-10f7e0475e2a admin admin] PATH_INFO: `/v3/projects/8c55f3d14ad14df5a59d0d43a21c7ed9/users/dd09716a091f4d98befa9909a8ad6a90/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:59.481538 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478c66cd-a71c-4a39-9db3-10f7e0475e2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c55f3d14ad14df5a59d0d43a21c7ed9, user_id=dd09716a091f4d98befa9909a8ad6a90, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:59.484661 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478c66cd-a71c-4a39-9db3-10f7e0475e2a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:59.520909 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 809/1605] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:06:59 2022] PUT /identity/v3/projects/dbb3b7f14b6647e1b250dba5d6aead39/users/2ed0f50aa4604d74ae75571524acfeb4/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:06:59.520909 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649599e-fbce-461f-af22-ea5dadbf5871 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:59.520909 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649599e-fbce-461f-af22-ea5dadbf5871 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:59.520909 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649599e-fbce-461f-af22-ea5dadbf5871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:59.528569 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b649599e-fbce-461f-af22-ea5dadbf5871 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:06:59.529779 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b649599e-fbce-461f-af22-ea5dadbf5871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:06:59.586378 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 810/1606] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:06:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:06:59.587214 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649599e-fbce-461f-af22-ea5dadbf5871 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:59.596329 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649599e-fbce-461f-af22-ea5dadbf5871 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:59.600886 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5cc77b-92ec-4910-a375-18bc2aa0b11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:59.601531 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5cc77b-92ec-4910-a375-18bc2aa0b11f admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:59.601808 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5cc77b-92ec-4910-a375-18bc2aa0b11f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:59.602076 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5cc77b-92ec-4910-a375-18bc2aa0b11f admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:59.602787 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5cc77b-92ec-4910-a375-18bc2aa0b11f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:59.604409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5cc77b-92ec-4910-a375-18bc2aa0b11f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:59.615577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 811/1607] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:06:59.616483 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5cc77b-92ec-4910-a375-18bc2aa0b11f admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:59.620187 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 797/1608] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:06:59 2022] PUT /identity/v3/projects/8c55f3d14ad14df5a59d0d43a21c7ed9/users/dd09716a091f4d98befa9909a8ad6a90/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:06:59.630994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c66cd-a71c-4a39-9db3-10f7e0475e2a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:59.634137 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5cc77b-92ec-4910-a375-18bc2aa0b11f admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:59.667424 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:06:59.698084 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73d47d-be7e-4761-9fdd-13d49c608ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:59.699117 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73d47d-be7e-4761-9fdd-13d49c608ba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:59.699470 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73d47d-be7e-4761-9fdd-13d49c608ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:59.699847 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73d47d-be7e-4761-9fdd-13d49c608ba3 admin admin] PATH_INFO: `/v3/projects/32672f6e771b4d37986e48e0091b6877/users/5645338e8194426eac77359c0cb233f5/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:59.710711 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c66cd-a71c-4a39-9db3-10f7e0475e2a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:59.715701 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882f2a0-6cee-443b-9c94-181a8166d66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:59.716785 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882f2a0-6cee-443b-9c94-181a8166d66f admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:59.716922 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73d47d-be7e-4761-9fdd-13d49c608ba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32672f6e771b4d37986e48e0091b6877, user_id=5645338e8194426eac77359c0cb233f5, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:59.717032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882f2a0-6cee-443b-9c94-181a8166d66f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:59.717285 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882f2a0-6cee-443b-9c94-181a8166d66f admin admin] PATH_INFO: `/v3/projects/7ffd8c68443f4bd6a3e7ace6b7b98cc0/users/c1892e46613f499ebe6c737860df82b3/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:59.719996 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73d47d-be7e-4761-9fdd-13d49c608ba3 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:59.733469 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8882f2a0-6cee-443b-9c94-181a8166d66f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ffd8c68443f4bd6a3e7ace6b7b98cc0, user_id=c1892e46613f499ebe6c737860df82b3, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:59.736349 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8882f2a0-6cee-443b-9c94-181a8166d66f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:59.820057 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 812/1609] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:06:59 2022] PUT /identity/v3/projects/32672f6e771b4d37986e48e0091b6877/users/5645338e8194426eac77359c0cb233f5/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:06:59.821395 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d25a1-33f2-45f2-bad9-4fe3e7b6eeb5 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:59.822061 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d25a1-33f2-45f2-bad9-4fe3e7b6eeb5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:59.822373 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d25a1-33f2-45f2-bad9-4fe3e7b6eeb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:59.838861 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 798/1610] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:06:59 2022] PUT /identity/v3/projects/7ffd8c68443f4bd6a3e7ace6b7b98cc0/users/c1892e46613f499ebe6c737860df82b3/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:06:59.849168 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882f2a0-6cee-443b-9c94-181a8166d66f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:59.852923 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9e4d25a1-33f2-45f2-bad9-4fe3e7b6eeb5 None None] MFA Rules not processed for user `bb1d18a000dd48d1b50569857c1783c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:06:59.854557 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9e4d25a1-33f2-45f2-bad9-4fe3e7b6eeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:06:59.919058 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882f2a0-6cee-443b-9c94-181a8166d66f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:59.924192 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d245a-9de0-4862-9675-bef40752782c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:59.925005 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d245a-9de0-4862-9675-bef40752782c admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:59.927524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d245a-9de0-4862-9675-bef40752782c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:59.927524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d245a-9de0-4862-9675-bef40752782c admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:06:59.927524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d245a-9de0-4862-9675-bef40752782c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:06:59.929784 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d245a-9de0-4862-9675-bef40752782c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:06:59.932124 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 813/1611] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:06:59 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:06:59.936233 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d25a1-33f2-45f2-bad9-4fe3e7b6eeb5 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:59.955184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 799/1612] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:06:59.955184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d245a-9de0-4862-9675-bef40752782c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:06:59.969231 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d245a-9de0-4862-9675-bef40752782c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:06:59.985232 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e4a28b-2a4c-484a-a2b3-59f06e0ad9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:06:59.986376 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4a28b-2a4c-484a-a2b3-59f06e0ad9d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:06:59.986675 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4a28b-2a4c-484a-a2b3-59f06e0ad9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:06:59.987091 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e4a28b-2a4c-484a-a2b3-59f06e0ad9d8 admin admin] PATH_INFO: `/v3/projects/71c9cca288b34019947e55d2a3112ddb/users/a3be90df779b4ddb8639d825fcb41234/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:00.010053 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e4a28b-2a4c-484a-a2b3-59f06e0ad9d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71c9cca288b34019947e55d2a3112ddb, user_id=a3be90df779b4ddb8639d825fcb41234, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:00.013822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e4a28b-2a4c-484a-a2b3-59f06e0ad9d8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:00.026607 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d25a1-33f2-45f2-bad9-4fe3e7b6eeb5 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:00.030665 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b3478-c717-4bca-9919-1d5a2e8f72fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:00.032267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b3478-c717-4bca-9919-1d5a2e8f72fa service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:00.032267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b3478-c717-4bca-9919-1d5a2e8f72fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:00.032267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b3478-c717-4bca-9919-1d5a2e8f72fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:00.042991 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b3478-c717-4bca-9919-1d5a2e8f72fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:00.044861 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b3478-c717-4bca-9919-1d5a2e8f72fa service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:00.056474 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 814/1613] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:06:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:00.058858 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171362c-7caf-420e-8fbb-4685a0a8603c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:00.058858 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171362c-7caf-420e-8fbb-4685a0a8603c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:00.058858 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171362c-7caf-420e-8fbb-4685a0a8603c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:00.078858 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 800/1614] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:06:59 2022] PUT /identity/v3/projects/71c9cca288b34019947e55d2a3112ddb/users/a3be90df779b4ddb8639d825fcb41234/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:00.080479 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a75db-5837-4cc3-b7dd-7b417bdbb401 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:00.082320 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a75db-5837-4cc3-b7dd-7b417bdbb401 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:00.082320 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a75db-5837-4cc3-b7dd-7b417bdbb401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:00.091553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d171362c-7caf-420e-8fbb-4685a0a8603c None None] MFA Rules not processed for user `dd09716a091f4d98befa9909a8ad6a90`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:00.092949 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d171362c-7caf-420e-8fbb-4685a0a8603c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:00.101792 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-936a75db-5837-4cc3-b7dd-7b417bdbb401 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:00.103800 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-936a75db-5837-4cc3-b7dd-7b417bdbb401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:00.109737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:00.170335 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:00.194679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 801/1615] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:00.211809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7867e51-0eb2-4a12-a2c9-75284df89751 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:00.211809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7867e51-0eb2-4a12-a2c9-75284df89751 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:00.211809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7867e51-0eb2-4a12-a2c9-75284df89751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:00.217619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 815/1616] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:00 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:00.217619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171362c-7caf-420e-8fbb-4685a0a8603c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:00.231013 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171362c-7caf-420e-8fbb-4685a0a8603c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:00.239174 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf48c25-25cf-45dc-a904-22e61bcaf840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:00.239341 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf48c25-25cf-45dc-a904-22e61bcaf840 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:00.239622 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf48c25-25cf-45dc-a904-22e61bcaf840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:00.239893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf48c25-25cf-45dc-a904-22e61bcaf840 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:00.241094 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf48c25-25cf-45dc-a904-22e61bcaf840 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:00.243362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf48c25-25cf-45dc-a904-22e61bcaf840 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:00.256273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f7867e51-0eb2-4a12-a2c9-75284df89751 None None] MFA Rules not processed for user `c1892e46613f499ebe6c737860df82b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:00.257646 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f7867e51-0eb2-4a12-a2c9-75284df89751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:00.330496 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 802/1617] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:00.331321 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7867e51-0eb2-4a12-a2c9-75284df89751 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:00.341223 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7867e51-0eb2-4a12-a2c9-75284df89751 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:00.345777 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d18bf-0961-4fd2-b2b0-4937c42286aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:00.346632 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d18bf-0961-4fd2-b2b0-4937c42286aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:00.346916 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d18bf-0961-4fd2-b2b0-4937c42286aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:00.347730 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d18bf-0961-4fd2-b2b0-4937c42286aa admin admin] PATH_INFO: `/v3/projects/dbb3b7f14b6647e1b250dba5d6aead39/users/2ed0f50aa4604d74ae75571524acfeb4/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:00.349764 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d18bf-0961-4fd2-b2b0-4937c42286aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbb3b7f14b6647e1b250dba5d6aead39, user_id=2ed0f50aa4604d74ae75571524acfeb4, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:00.352467 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d18bf-0961-4fd2-b2b0-4937c42286aa admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:00.359904 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 816/1618] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:07:00 2022] POST /identity/v3/projects => generated 367 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:00.362548 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f43ca-b024-4366-9baa-bdaeadf42bbc None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:00.362548 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f43ca-b024-4366-9baa-bdaeadf42bbc None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:00.362548 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f43ca-b024-4366-9baa-bdaeadf42bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:00.365261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 803/1619] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:00 2022] PUT /identity/v3/projects/dbb3b7f14b6647e1b250dba5d6aead39/users/2ed0f50aa4604d74ae75571524acfeb4/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:00.366045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d18bf-0961-4fd2-b2b0-4937c42286aa admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:00.390772 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-176f43ca-b024-4366-9baa-bdaeadf42bbc None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:00.392065 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-176f43ca-b024-4366-9baa-bdaeadf42bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:00.456857 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d18bf-0961-4fd2-b2b0-4937c42286aa admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:00.464892 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a4b05-6d8f-4787-9c14-47620880ecdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:00.466365 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a4b05-6d8f-4787-9c14-47620880ecdb admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:00.466365 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a4b05-6d8f-4787-9c14-47620880ecdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:00.466561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37a4b05-6d8f-4787-9c14-47620880ecdb admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:00.467874 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a4b05-6d8f-4787-9c14-47620880ecdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:00.468895 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 817/1620] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:00.472839 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37a4b05-6d8f-4787-9c14-47620880ecdb admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:00.473015 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f43ca-b024-4366-9baa-bdaeadf42bbc None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:00.473015 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-176f43ca-b024-4366-9baa-bdaeadf42bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:00.555941 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f43ca-b024-4366-9baa-bdaeadf42bbc None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:00.560416 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 804/1621] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:07:00 2022] POST /identity/v3/projects => generated 363 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:00.560416 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a4b05-6d8f-4787-9c14-47620880ecdb admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:00.562004 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a294504-1531-49e6-a26b-26cebc67a38c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:00.562727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a294504-1531-49e6-a26b-26cebc67a38c service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:00.563088 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a294504-1531-49e6-a26b-26cebc67a38c service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:00.563425 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a294504-1531-49e6-a26b-26cebc67a38c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:00.574343 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a294504-1531-49e6-a26b-26cebc67a38c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:00.575979 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a294504-1531-49e6-a26b-26cebc67a38c service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:00.576142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:00.587280 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 818/1622] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:00.588395 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a294504-1531-49e6-a26b-26cebc67a38c service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:00.601670 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a294504-1531-49e6-a26b-26cebc67a38c service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:00.635298 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37a4b05-6d8f-4787-9c14-47620880ecdb admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:00.649882 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c31bc-b59f-4861-84b6-b461cd6438f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:00.650739 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c31bc-b59f-4861-84b6-b461cd6438f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:00.651224 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c31bc-b59f-4861-84b6-b461cd6438f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:00.651513 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c31bc-b59f-4861-84b6-b461cd6438f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:00.652218 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1c31bc-b59f-4861-84b6-b461cd6438f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:00.655648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1c31bc-b59f-4861-84b6-b461cd6438f2 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:00.672159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea8a90-8aa4-4606-b932-916e0854b0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:00.672907 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea8a90-8aa4-4606-b932-916e0854b0c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:00.673684 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea8a90-8aa4-4606-b932-916e0854b0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:00.673684 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea8a90-8aa4-4606-b932-916e0854b0c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:00.674465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea8a90-8aa4-4606-b932-916e0854b0c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:00.676781 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea8a90-8aa4-4606-b932-916e0854b0c6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:00.686193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:00.806840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 819/1623] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:07:00 2022] POST /identity/v3/users => generated 290 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:00.806840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53687c-ebe7-4e83-8dc4-33c7924f9249 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:00.806840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53687c-ebe7-4e83-8dc4-33c7924f9249 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:00.807303 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 805/1624] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:07:00 2022] POST /identity/v3/projects => generated 385 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:00.807505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c31bc-b59f-4861-84b6-b461cd6438f2 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:00.807624 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c53687c-ebe7-4e83-8dc4-33c7924f9249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:00.846152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8c53687c-ebe7-4e83-8dc4-33c7924f9249 None None] MFA Rules not processed for user `2ed0f50aa4604d74ae75571524acfeb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:00.848553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8c53687c-ebe7-4e83-8dc4-33c7924f9249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:00.891959 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c31bc-b59f-4861-84b6-b461cd6438f2 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:00.896456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72589ed8-8a58-4909-b9bf-10bbb3af6803 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:00.897211 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72589ed8-8a58-4909-b9bf-10bbb3af6803 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:00.897605 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72589ed8-8a58-4909-b9bf-10bbb3af6803 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:00.898030 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72589ed8-8a58-4909-b9bf-10bbb3af6803 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:00.956293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 820/1625] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:00 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:00.957371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53687c-ebe7-4e83-8dc4-33c7924f9249 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:00.984974 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72589ed8-8a58-4909-b9bf-10bbb3af6803 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:00.986842 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72589ed8-8a58-4909-b9bf-10bbb3af6803 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:00.998865 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 806/1626] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:00.999753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72589ed8-8a58-4909-b9bf-10bbb3af6803 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:01.012223 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72589ed8-8a58-4909-b9bf-10bbb3af6803 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:01.019851 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfbb6c-b174-4904-ac7f-de2fc73b4a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:01.020659 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfbb6c-b174-4904-ac7f-de2fc73b4a41 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:01.021018 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfbb6c-b174-4904-ac7f-de2fc73b4a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:01.021366 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfbb6c-b174-4904-ac7f-de2fc73b4a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:01.022172 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcfbb6c-b174-4904-ac7f-de2fc73b4a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:01.024021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcfbb6c-b174-4904-ac7f-de2fc73b4a41 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:01.040762 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 807/1627] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:01.040762 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfbb6c-b174-4904-ac7f-de2fc73b4a41 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:01.050068 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c53687c-ebe7-4e83-8dc4-33c7924f9249 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:01.052253 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfbb6c-b174-4904-ac7f-de2fc73b4a41 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:01.058085 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7924a-18df-458e-ad9e-fa2763ead675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:01.058272 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732c94b-34fb-4e0d-8a59-6d3dc9c733b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:01.058710 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7924a-18df-458e-ad9e-fa2763ead675 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:01.058996 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7924a-18df-458e-ad9e-fa2763ead675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:01.059122 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732c94b-34fb-4e0d-8a59-6d3dc9c733b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:01.059318 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7924a-18df-458e-ad9e-fa2763ead675 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:01.059612 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732c94b-34fb-4e0d-8a59-6d3dc9c733b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:01.059973 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732c94b-34fb-4e0d-8a59-6d3dc9c733b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:01.060096 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7924a-18df-458e-ad9e-fa2763ead675 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:01.060875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732c94b-34fb-4e0d-8a59-6d3dc9c733b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:01.062323 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7924a-18df-458e-ad9e-fa2763ead675 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:01.063102 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b732c94b-34fb-4e0d-8a59-6d3dc9c733b6 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:01.185659 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 821/1628] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:07:00 2022] POST /identity/v3/users => generated 288 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:01.185659 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7924a-18df-458e-ad9e-fa2763ead675 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:01.209427 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 808/1629] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:07:01 2022] POST /identity/v3/users => generated 299 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:01.210509 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732c94b-34fb-4e0d-8a59-6d3dc9c733b6 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:01.225155 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732c94b-34fb-4e0d-8a59-6d3dc9c733b6 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:01.236834 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47248030-5ca9-4a71-9560-c0bf2c46c647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:01.237152 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47248030-5ca9-4a71-9560-c0bf2c46c647 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:01.238859 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47248030-5ca9-4a71-9560-c0bf2c46c647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:01.239248 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47248030-5ca9-4a71-9560-c0bf2c46c647 admin admin] PATH_INFO: `/v3/projects/0999954e63ea461f91ff7ddf6fb60086/users/24bcfd9934b049999addbd0246cf03ed/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:01.276480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7924a-18df-458e-ad9e-fa2763ead675 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:01.284390 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9e49e-ce88-4327-8330-da31ce60dfca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:01.285177 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47248030-5ca9-4a71-9560-c0bf2c46c647 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0999954e63ea461f91ff7ddf6fb60086, user_id=24bcfd9934b049999addbd0246cf03ed, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:01.286014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9e49e-ce88-4327-8330-da31ce60dfca service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:01.286503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9e49e-ce88-4327-8330-da31ce60dfca service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:01.286944 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9e49e-ce88-4327-8330-da31ce60dfca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:01.287865 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47248030-5ca9-4a71-9560-c0bf2c46c647 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:01.293535 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:01.304872 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e9e49e-ce88-4327-8330-da31ce60dfca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:01.306933 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e9e49e-ce88-4327-8330-da31ce60dfca service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:01.325589 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 822/1630] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:01 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:01.326483 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9e49e-ce88-4327-8330-da31ce60dfca service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:01.341560 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9e49e-ce88-4327-8330-da31ce60dfca service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:01.357232 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 809/1631] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:01 2022] PUT /identity/v3/projects/0999954e63ea461f91ff7ddf6fb60086/users/24bcfd9934b049999addbd0246cf03ed/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:01.357232 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47248030-5ca9-4a71-9560-c0bf2c46c647 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:01.361850 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e06e85-239a-410f-8567-02cb1f64752d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:01.362592 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e06e85-239a-410f-8567-02cb1f64752d admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:01.364380 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e06e85-239a-410f-8567-02cb1f64752d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:01.364723 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e06e85-239a-410f-8567-02cb1f64752d admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:01.365481 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e06e85-239a-410f-8567-02cb1f64752d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:01.367376 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e06e85-239a-410f-8567-02cb1f64752d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:01.369023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:01.380494 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 823/1632] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:01.381433 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e06e85-239a-410f-8567-02cb1f64752d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:01.400723 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e06e85-239a-410f-8567-02cb1f64752d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:01.437488 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47248030-5ca9-4a71-9560-c0bf2c46c647 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:01.464247 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7199a07-9bcc-4e48-a77d-2abcec60c073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:01.465272 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7199a07-9bcc-4e48-a77d-2abcec60c073 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:01.468279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7199a07-9bcc-4e48-a77d-2abcec60c073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:01.470619 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7199a07-9bcc-4e48-a77d-2abcec60c073 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:01.470619 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7199a07-9bcc-4e48-a77d-2abcec60c073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:01.477989 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7199a07-9bcc-4e48-a77d-2abcec60c073 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:01.502597 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b2cd0-f9df-4bce-9ec1-41bad7d397d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:01.503328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b2cd0-f9df-4bce-9ec1-41bad7d397d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:01.503616 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b2cd0-f9df-4bce-9ec1-41bad7d397d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:01.503896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b2cd0-f9df-4bce-9ec1-41bad7d397d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:01.504687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623b2cd0-f9df-4bce-9ec1-41bad7d397d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:01.506519 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623b2cd0-f9df-4bce-9ec1-41bad7d397d1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:01.515940 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 810/1633] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:01.519699 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7199a07-9bcc-4e48-a77d-2abcec60c073 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:01.523915 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 824/1634] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:01.523915 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b2cd0-f9df-4bce-9ec1-41bad7d397d1 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:01.534319 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b2cd0-f9df-4bce-9ec1-41bad7d397d1 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:01.540509 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c941e-3d35-421b-a3d7-2cf5d205f291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:01.541418 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c941e-3d35-421b-a3d7-2cf5d205f291 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:01.541706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c941e-3d35-421b-a3d7-2cf5d205f291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:01.541988 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c941e-3d35-421b-a3d7-2cf5d205f291 admin admin] PATH_INFO: `/v3/projects/7bd65d65021c430b86b9af5eaf4eb59e/users/9eb1f185c33648459e9d1c547c4084f1/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:01.544770 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7199a07-9bcc-4e48-a77d-2abcec60c073 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:01.556949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31362359-af14-46fc-903f-efbac3d7566f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:01.560711 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31362359-af14-46fc-903f-efbac3d7566f admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:01.561546 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31362359-af14-46fc-903f-efbac3d7566f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:01.562270 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31362359-af14-46fc-903f-efbac3d7566f admin admin] PATH_INFO: `/v3/projects/e208585b066d4699b288644e48300e7f/users/2066ab7ad8e04aac83b18b50c79a09c3/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:01.563524 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c941e-3d35-421b-a3d7-2cf5d205f291 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bd65d65021c430b86b9af5eaf4eb59e, user_id=9eb1f185c33648459e9d1c547c4084f1, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:01.567363 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c941e-3d35-421b-a3d7-2cf5d205f291 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:01.602967 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31362359-af14-46fc-903f-efbac3d7566f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e208585b066d4699b288644e48300e7f, user_id=2066ab7ad8e04aac83b18b50c79a09c3, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:01.605186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31362359-af14-46fc-903f-efbac3d7566f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:01.658526 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 825/1635] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:01 2022] PUT /identity/v3/projects/7bd65d65021c430b86b9af5eaf4eb59e/users/9eb1f185c33648459e9d1c547c4084f1/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:01.658526 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c941e-3d35-421b-a3d7-2cf5d205f291 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:01.706212 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 811/1636] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:01 2022] PUT /identity/v3/projects/e208585b066d4699b288644e48300e7f/users/2066ab7ad8e04aac83b18b50c79a09c3/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:01.710880 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31362359-af14-46fc-903f-efbac3d7566f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:01.745150 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c941e-3d35-421b-a3d7-2cf5d205f291 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:01.806721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31362359-af14-46fc-903f-efbac3d7566f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:01.817027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4322ba-e041-4491-8fd7-4bfb03d1c134 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:01.823846 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4322ba-e041-4491-8fd7-4bfb03d1c134 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:01.823846 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4322ba-e041-4491-8fd7-4bfb03d1c134 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:01.823846 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4322ba-e041-4491-8fd7-4bfb03d1c134 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:01.850315 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f409f-99e6-44cc-bad9-62e9d8aed684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:01.852926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f409f-99e6-44cc-bad9-62e9d8aed684 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:01.852926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f409f-99e6-44cc-bad9-62e9d8aed684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:01.852926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f409f-99e6-44cc-bad9-62e9d8aed684 admin admin] PATH_INFO: `/v3/projects/0999954e63ea461f91ff7ddf6fb60086/users/24bcfd9934b049999addbd0246cf03ed/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:01.857791 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f409f-99e6-44cc-bad9-62e9d8aed684 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0999954e63ea461f91ff7ddf6fb60086, user_id=24bcfd9934b049999addbd0246cf03ed, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:01.862369 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f409f-99e6-44cc-bad9-62e9d8aed684 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:01.884386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 826/1637] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:01 2022] PUT /identity/v3/projects/0999954e63ea461f91ff7ddf6fb60086/users/24bcfd9934b049999addbd0246cf03ed/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:01.886954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f409f-99e6-44cc-bad9-62e9d8aed684 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:01.892946 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4322ba-e041-4491-8fd7-4bfb03d1c134 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:01.895477 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4322ba-e041-4491-8fd7-4bfb03d1c134 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:01.928400 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 812/1638] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:01.930176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4322ba-e041-4491-8fd7-4bfb03d1c134 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:01.963788 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4322ba-e041-4491-8fd7-4bfb03d1c134 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:02.134626 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:02.135051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f409f-99e6-44cc-bad9-62e9d8aed684 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:02.135051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4569b-764b-44c4-8c0e-76b88df1bb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:02.135051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4569b-764b-44c4-8c0e-76b88df1bb28 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:02.135051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4569b-764b-44c4-8c0e-76b88df1bb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:02.135544 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4569b-764b-44c4-8c0e-76b88df1bb28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:02.135544 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d4569b-764b-44c4-8c0e-76b88df1bb28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:02.135544 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d4569b-764b-44c4-8c0e-76b88df1bb28 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:02.135544 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 827/1639] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:02.135544 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0ecda-1fd7-4291-a53c-b6e0df8db752 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:02.135544 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0ecda-1fd7-4291-a53c-b6e0df8db752 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:02.136007 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0ecda-1fd7-4291-a53c-b6e0df8db752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:02.140224 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1685d87-1a66-42cd-85a3-c9dda5f0a586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:02.149915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1685d87-1a66-42cd-85a3-c9dda5f0a586 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:02.149915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1685d87-1a66-42cd-85a3-c9dda5f0a586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:02.149915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1685d87-1a66-42cd-85a3-c9dda5f0a586 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:02.149915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1685d87-1a66-42cd-85a3-c9dda5f0a586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:02.149915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1685d87-1a66-42cd-85a3-c9dda5f0a586 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:02.158947 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 813/1640] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:02.160226 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1685d87-1a66-42cd-85a3-c9dda5f0a586 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:02.172345 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1685d87-1a66-42cd-85a3-c9dda5f0a586 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:02.178904 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1fa4a-3bdb-43f3-a86d-18295b5a2516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:02.180553 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1fa4a-3bdb-43f3-a86d-18295b5a2516 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:02.180553 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1fa4a-3bdb-43f3-a86d-18295b5a2516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:02.180553 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1fa4a-3bdb-43f3-a86d-18295b5a2516 admin admin] PATH_INFO: `/v3/projects/7bd65d65021c430b86b9af5eaf4eb59e/users/9eb1f185c33648459e9d1c547c4084f1/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:02.182654 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a1fa4a-3bdb-43f3-a86d-18295b5a2516 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bd65d65021c430b86b9af5eaf4eb59e, user_id=9eb1f185c33648459e9d1c547c4084f1, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:02.185438 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a1fa4a-3bdb-43f3-a86d-18295b5a2516 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:02.198373 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e1c0ecda-1fd7-4291-a53c-b6e0df8db752 None None] MFA Rules not processed for user `24bcfd9934b049999addbd0246cf03ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:02.209256 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 814/1641] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:02 2022] PUT /identity/v3/projects/7bd65d65021c430b86b9af5eaf4eb59e/users/9eb1f185c33648459e9d1c547c4084f1/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:02.209490 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e1c0ecda-1fd7-4291-a53c-b6e0df8db752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:02.209620 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1fa4a-3bdb-43f3-a86d-18295b5a2516 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:02.270754 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:02.336449 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1fa4a-3bdb-43f3-a86d-18295b5a2516 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:02.344027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c6829-561d-45e9-b076-824f2d1f70fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:02.345097 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c6829-561d-45e9-b076-824f2d1f70fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:02.345400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 828/1642] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:02 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:02.347116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f456b2-3cef-4d1f-9305-dca23819d58c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:02.347985 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f456b2-3cef-4d1f-9305-dca23819d58c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:02.348273 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f456b2-3cef-4d1f-9305-dca23819d58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:02.349635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c6829-561d-45e9-b076-824f2d1f70fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:02.350045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c6829-561d-45e9-b076-824f2d1f70fe admin admin] PATH_INFO: `/v3/projects/e208585b066d4699b288644e48300e7f/users/2066ab7ad8e04aac83b18b50c79a09c3/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:02.352732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c6829-561d-45e9-b076-824f2d1f70fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e208585b066d4699b288644e48300e7f, user_id=2066ab7ad8e04aac83b18b50c79a09c3, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:02.355684 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c6829-561d-45e9-b076-824f2d1f70fe admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:02.374046 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 815/1643] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:02 2022] PUT /identity/v3/projects/e208585b066d4699b288644e48300e7f/users/2066ab7ad8e04aac83b18b50c79a09c3/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:02.375047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c6829-561d-45e9-b076-824f2d1f70fe admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:02.413326 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-81f456b2-3cef-4d1f-9305-dca23819d58c None None] MFA Rules not processed for user `9eb1f185c33648459e9d1c547c4084f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:02.413326 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-81f456b2-3cef-4d1f-9305-dca23819d58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:02.456654 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c6829-561d-45e9-b076-824f2d1f70fe admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:02.461715 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9493aa0-b8b7-422f-b0c6-ddcf04ce33b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:02.462730 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9493aa0-b8b7-422f-b0c6-ddcf04ce33b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:02.464062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9493aa0-b8b7-422f-b0c6-ddcf04ce33b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:02.464610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9493aa0-b8b7-422f-b0c6-ddcf04ce33b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:02.555051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9493aa0-b8b7-422f-b0c6-ddcf04ce33b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:02.556857 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9493aa0-b8b7-422f-b0c6-ddcf04ce33b8 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:02.558156 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 829/1644] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:02.559709 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980a62e-d9f9-47cc-9184-2cdacae3186c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:02.561431 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980a62e-d9f9-47cc-9184-2cdacae3186c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:02.561885 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980a62e-d9f9-47cc-9184-2cdacae3186c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:02.577589 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 816/1645] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:02 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:02.604483 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b980a62e-d9f9-47cc-9184-2cdacae3186c None None] MFA Rules not processed for user `2066ab7ad8e04aac83b18b50c79a09c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:02.606394 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b980a62e-d9f9-47cc-9184-2cdacae3186c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:02.624459 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9493aa0-b8b7-422f-b0c6-ddcf04ce33b8 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:02.639597 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9493aa0-b8b7-422f-b0c6-ddcf04ce33b8 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:02.650596 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532cf11e-3b8f-4f15-9688-8fa3544c5aad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:02.658509 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532cf11e-3b8f-4f15-9688-8fa3544c5aad service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:02.659218 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532cf11e-3b8f-4f15-9688-8fa3544c5aad service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:02.660281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532cf11e-3b8f-4f15-9688-8fa3544c5aad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:02.676255 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532cf11e-3b8f-4f15-9688-8fa3544c5aad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:02.678008 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532cf11e-3b8f-4f15-9688-8fa3544c5aad service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:02.693503 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 817/1646] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:02 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 20:07:02.814992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 830/1647] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:02 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:02.935726 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532cf11e-3b8f-4f15-9688-8fa3544c5aad service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:02.940649 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980a62e-d9f9-47cc-9184-2cdacae3186c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:02.951385 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980a62e-d9f9-47cc-9184-2cdacae3186c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:02.955679 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d31e1-f7d2-4a0e-ad93-edf8588531e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:02.957093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d31e1-f7d2-4a0e-ad93-edf8588531e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:02.957436 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d31e1-f7d2-4a0e-ad93-edf8588531e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:02.957716 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d31e1-f7d2-4a0e-ad93-edf8588531e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:02.969818 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105d31e1-f7d2-4a0e-ad93-edf8588531e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:02.971389 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105d31e1-f7d2-4a0e-ad93-edf8588531e7 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:02.984510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 831/1648] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:02 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:03.021342 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d31e1-f7d2-4a0e-ad93-edf8588531e7 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:03.035483 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d31e1-f7d2-4a0e-ad93-edf8588531e7 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:03.040379 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532cf11e-3b8f-4f15-9688-8fa3544c5aad service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:03.047040 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210daee-bf31-4b09-880e-28ec3277e568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:03.048168 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210daee-bf31-4b09-880e-28ec3277e568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:03.048483 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210daee-bf31-4b09-880e-28ec3277e568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:03.048859 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210daee-bf31-4b09-880e-28ec3277e568 admin admin] PATH_INFO: `/v3/users/c1892e46613f499ebe6c737860df82b3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:03.050631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210daee-bf31-4b09-880e-28ec3277e568 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1892e46613f499ebe6c737860df82b3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:03.051225 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272039be-2737-4d74-9065-e3df2ac78bdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:03.052099 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272039be-2737-4d74-9065-e3df2ac78bdf service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:03.052491 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272039be-2737-4d74-9065-e3df2ac78bdf service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:03.052635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: WARNING py.warnings [None req-2210daee-bf31-4b09-880e-28ec3277e568 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 20:07:03.052635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: warnings.warn(msg) Jun 23 20:07:03.052635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: Jun 23 20:07:03.053291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272039be-2737-4d74-9065-e3df2ac78bdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:03.054028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210daee-bf31-4b09-880e-28ec3277e568 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:03.192549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272039be-2737-4d74-9065-e3df2ac78bdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:03.194591 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272039be-2737-4d74-9065-e3df2ac78bdf service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:03.210128 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 832/1649] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:03 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 20:07:03.251008 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272039be-2737-4d74-9065-e3df2ac78bdf service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:03.262012 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272039be-2737-4d74-9065-e3df2ac78bdf service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:03.286611 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586573ab-5354-4e8f-9a46-0e0e4150cb06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:03.287277 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586573ab-5354-4e8f-9a46-0e0e4150cb06 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:03.287558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586573ab-5354-4e8f-9a46-0e0e4150cb06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:03.288093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586573ab-5354-4e8f-9a46-0e0e4150cb06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:03.296871 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2210daee-bf31-4b09-880e-28ec3277e568 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1892e46613f499ebe6c737860df82b3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:03.298116 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2210daee-bf31-4b09-880e-28ec3277e568 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1892e46613f499ebe6c737860df82b3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:03.383038 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2210daee-bf31-4b09-880e-28ec3277e568 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1892e46613f499ebe6c737860df82b3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:03.395900 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:03.424080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2210daee-bf31-4b09-880e-28ec3277e568 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1892e46613f499ebe6c737860df82b3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:03.512885 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586573ab-5354-4e8f-9a46-0e0e4150cb06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:03.823586 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586573ab-5354-4e8f-9a46-0e0e4150cb06 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:03.823586 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 833/1650] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:03 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 20:07:03.824098 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2210daee-bf31-4b09-880e-28ec3277e568 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1892e46613f499ebe6c737860df82b3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:03.824098 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 818/1651] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:07:02 2022] DELETE /identity/v3/users/c1892e46613f499ebe6c737860df82b3 => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:03.824587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe6709-c359-40b6-a4d5-1b7c3ce8bef2 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:03.824587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe6709-c359-40b6-a4d5-1b7c3ce8bef2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:03.824587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe6709-c359-40b6-a4d5-1b7c3ce8bef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:03.824587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:03.824587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-99fe6709-c359-40b6-a4d5-1b7c3ce8bef2 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:03.824587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-99fe6709-c359-40b6-a4d5-1b7c3ce8bef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:03.825478 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 834/1652] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:03.825855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210daee-bf31-4b09-880e-28ec3277e568 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:03.825855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2210daee-bf31-4b09-880e-28ec3277e568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:03.840109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210daee-bf31-4b09-880e-28ec3277e568 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:03.845649 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f801e6-9b13-417a-b690-afe469f53505 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:03.846488 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f801e6-9b13-417a-b690-afe469f53505 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:03.846779 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f801e6-9b13-417a-b690-afe469f53505 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:03.847151 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f801e6-9b13-417a-b690-afe469f53505 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:03.859175 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f801e6-9b13-417a-b690-afe469f53505 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:03.860680 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f801e6-9b13-417a-b690-afe469f53505 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:03.874447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 819/1653] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:04.554396 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:05.060085 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:05.622320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe6709-c359-40b6-a4d5-1b7c3ce8bef2 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:05.623795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-99fe6709-c359-40b6-a4d5-1b7c3ce8bef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:05.659953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe6709-c359-40b6-a4d5-1b7c3ce8bef2 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:05.680728 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e67e2a-ec2e-4f7f-a79a-3ff7de63850b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:05.680728 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e67e2a-ec2e-4f7f-a79a-3ff7de63850b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:05.680728 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e67e2a-ec2e-4f7f-a79a-3ff7de63850b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:05.680728 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e67e2a-ec2e-4f7f-a79a-3ff7de63850b admin admin] PATH_INFO: `/v3/projects/7ffd8c68443f4bd6a3e7ace6b7b98cc0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:05.681797 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e67e2a-ec2e-4f7f-a79a-3ff7de63850b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ffd8c68443f4bd6a3e7ace6b7b98cc0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:05.682831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: WARNING py.warnings [None req-d1e67e2a-ec2e-4f7f-a79a-3ff7de63850b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 20:07:05.682831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: warnings.warn(msg) Jun 23 20:07:05.682831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: Jun 23 20:07:05.684197 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e67e2a-ec2e-4f7f-a79a-3ff7de63850b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:05.896905 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d1e67e2a-ec2e-4f7f-a79a-3ff7de63850b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ffd8c68443f4bd6a3e7ace6b7b98cc0', 'request_id': 'req-d1e67e2a-ec2e-4f7f-a79a-3ff7de63850b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:06.025071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d1e67e2a-ec2e-4f7f-a79a-3ff7de63850b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ffd8c68443f4bd6a3e7ace6b7b98cc0', 'request_id': 'req-d1e67e2a-ec2e-4f7f-a79a-3ff7de63850b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:06.052404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d1e67e2a-ec2e-4f7f-a79a-3ff7de63850b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ffd8c68443f4bd6a3e7ace6b7b98cc0', 'request_id': 'req-d1e67e2a-ec2e-4f7f-a79a-3ff7de63850b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:06.089528 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d1e67e2a-ec2e-4f7f-a79a-3ff7de63850b admin admin] The token cache is being invalidate because project 7ffd8c68443f4bd6a3e7ace6b7b98cc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:07:06.090511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d1e67e2a-ec2e-4f7f-a79a-3ff7de63850b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:06.091416 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d1e67e2a-ec2e-4f7f-a79a-3ff7de63850b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:06.094521 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 835/1654] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:07:05 2022] DELETE /identity/v3/projects/7ffd8c68443f4bd6a3e7ace6b7b98cc0 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:06.099027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f801e6-9b13-417a-b690-afe469f53505 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:06.109319 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f801e6-9b13-417a-b690-afe469f53505 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:06.278263 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8dba1-7960-43cf-b299-24c448bb335c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:06.288658 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8dba1-7960-43cf-b299-24c448bb335c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:06.289935 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8dba1-7960-43cf-b299-24c448bb335c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:06.292108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8dba1-7960-43cf-b299-24c448bb335c admin admin] PATH_INFO: `/v3/users/9eb1f185c33648459e9d1c547c4084f1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:06.296182 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8dba1-7960-43cf-b299-24c448bb335c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eb1f185c33648459e9d1c547c4084f1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:06.315244 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8dba1-7960-43cf-b299-24c448bb335c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:06.714275 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-70c8dba1-7960-43cf-b299-24c448bb335c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eb1f185c33648459e9d1c547c4084f1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:06.719895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-70c8dba1-7960-43cf-b299-24c448bb335c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eb1f185c33648459e9d1c547c4084f1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:06.803300 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-70c8dba1-7960-43cf-b299-24c448bb335c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eb1f185c33648459e9d1c547c4084f1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:06.856456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-70c8dba1-7960-43cf-b299-24c448bb335c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eb1f185c33648459e9d1c547c4084f1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:06.973987 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-70c8dba1-7960-43cf-b299-24c448bb335c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eb1f185c33648459e9d1c547c4084f1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:06.981123 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 820/1655] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:07:06 2022] DELETE /identity/v3/users/9eb1f185c33648459e9d1c547c4084f1 => generated 0 bytes in 882 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:09.134067 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e67e2a-ec2e-4f7f-a79a-3ff7de63850b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:09.136163 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d1e67e2a-ec2e-4f7f-a79a-3ff7de63850b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:09.167154 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e67e2a-ec2e-4f7f-a79a-3ff7de63850b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:09.367227 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07fbec-75c5-4577-9420-a96c440e3590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:09.368367 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07fbec-75c5-4577-9420-a96c440e3590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:09.368790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07fbec-75c5-4577-9420-a96c440e3590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:09.369225 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07fbec-75c5-4577-9420-a96c440e3590 admin admin] PATH_INFO: `/v3/projects/7bd65d65021c430b86b9af5eaf4eb59e` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:09.370611 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f07fbec-75c5-4577-9420-a96c440e3590 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bd65d65021c430b86b9af5eaf4eb59e)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:09.376974 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f07fbec-75c5-4577-9420-a96c440e3590 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:09.599815 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3f07fbec-75c5-4577-9420-a96c440e3590 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bd65d65021c430b86b9af5eaf4eb59e', 'request_id': 'req-3f07fbec-75c5-4577-9420-a96c440e3590'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:09.601983 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:09.707770 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3f07fbec-75c5-4577-9420-a96c440e3590 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bd65d65021c430b86b9af5eaf4eb59e', 'request_id': 'req-3f07fbec-75c5-4577-9420-a96c440e3590'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:09.714817 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3f07fbec-75c5-4577-9420-a96c440e3590 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bd65d65021c430b86b9af5eaf4eb59e', 'request_id': 'req-3f07fbec-75c5-4577-9420-a96c440e3590'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:09.752334 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3f07fbec-75c5-4577-9420-a96c440e3590 admin admin] The token cache is being invalidate because project 7bd65d65021c430b86b9af5eaf4eb59e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:07:09.752605 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3f07fbec-75c5-4577-9420-a96c440e3590 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:09.755242 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3f07fbec-75c5-4577-9420-a96c440e3590 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:09.759129 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 836/1656] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:07:09 2022] DELETE /identity/v3/projects/7bd65d65021c430b86b9af5eaf4eb59e => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:09.764333 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4c1cb-4721-4b82-84d9-404539315aff None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:09.765279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4c1cb-4721-4b82-84d9-404539315aff None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:09.765724 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf4c1cb-4721-4b82-84d9-404539315aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:09.844147 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-fdf4c1cb-4721-4b82-84d9-404539315aff None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:09.847705 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fdf4c1cb-4721-4b82-84d9-404539315aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:10.085290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07fbec-75c5-4577-9420-a96c440e3590 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:10.086762 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3f07fbec-75c5-4577-9420-a96c440e3590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:10.130648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 821/1657] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:10.136625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ee165-4c25-4092-b584-5c2a26074db7 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:10.136917 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ee165-4c25-4092-b584-5c2a26074db7 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:10.137192 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ee165-4c25-4092-b584-5c2a26074db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:10.185263 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-3d1ee165-4c25-4092-b584-5c2a26074db7 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:10.186541 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3d1ee165-4c25-4092-b584-5c2a26074db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:10.219870 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 822/1658] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:10.226195 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ee165-4c25-4092-b584-5c2a26074db7 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:10.246783 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ee165-4c25-4092-b584-5c2a26074db7 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:10.252175 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c552565a-083f-4561-964b-5bf9fe8d5374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:10.253329 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552565a-083f-4561-964b-5bf9fe8d5374 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:10.253754 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552565a-083f-4561-964b-5bf9fe8d5374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:10.254169 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552565a-083f-4561-964b-5bf9fe8d5374 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:10.255091 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c552565a-083f-4561-964b-5bf9fe8d5374 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:07:10.255550 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c552565a-083f-4561-964b-5bf9fe8d5374 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:10.257659 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c552565a-083f-4561-964b-5bf9fe8d5374 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:10.268311 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:10.299451 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07fbec-75c5-4577-9420-a96c440e3590 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:10.310331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 823/1659] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:07:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:07:10.311724 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c552565a-083f-4561-964b-5bf9fe8d5374 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:10.318652 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6bf2e-87b1-4f97-b1e3-cead20b2d215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:10.322579 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6bf2e-87b1-4f97-b1e3-cead20b2d215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:10.322579 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6bf2e-87b1-4f97-b1e3-cead20b2d215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:10.322579 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6bf2e-87b1-4f97-b1e3-cead20b2d215 admin admin] PATH_INFO: `/v3/users/dd09716a091f4d98befa9909a8ad6a90` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:10.323529 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6bf2e-87b1-4f97-b1e3-cead20b2d215 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd09716a091f4d98befa9909a8ad6a90)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:10.325681 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6bf2e-87b1-4f97-b1e3-cead20b2d215 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:10.353558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c552565a-083f-4561-964b-5bf9fe8d5374 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:10.367113 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9342b5ee-b648-4293-b43c-bdeaa5b54eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:10.370332 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342b5ee-b648-4293-b43c-bdeaa5b54eb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:10.370553 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342b5ee-b648-4293-b43c-bdeaa5b54eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:10.370808 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342b5ee-b648-4293-b43c-bdeaa5b54eb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:10.371537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9342b5ee-b648-4293-b43c-bdeaa5b54eb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:10.378983 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9342b5ee-b648-4293-b43c-bdeaa5b54eb0 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:10.540776 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 824/1660] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:07:10 2022] POST /identity/v3/projects => generated 379 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:10.541765 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9342b5ee-b648-4293-b43c-bdeaa5b54eb0 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:10.619704 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dfb6bf2e-87b1-4f97-b1e3-cead20b2d215 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd09716a091f4d98befa9909a8ad6a90'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:10.620669 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dfb6bf2e-87b1-4f97-b1e3-cead20b2d215 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd09716a091f4d98befa9909a8ad6a90'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:10.681634 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9342b5ee-b648-4293-b43c-bdeaa5b54eb0 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:10.695920 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ac025-b823-47a9-ad51-049f8275ef36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:10.697401 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ac025-b823-47a9-ad51-049f8275ef36 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:10.699190 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ac025-b823-47a9-ad51-049f8275ef36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:10.700038 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ac025-b823-47a9-ad51-049f8275ef36 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:10.701024 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ac025-b823-47a9-ad51-049f8275ef36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:10.708982 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ac025-b823-47a9-ad51-049f8275ef36 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:10.720489 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dfb6bf2e-87b1-4f97-b1e3-cead20b2d215 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd09716a091f4d98befa9909a8ad6a90'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:10.720489 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dfb6bf2e-87b1-4f97-b1e3-cead20b2d215 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd09716a091f4d98befa9909a8ad6a90'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:10.746875 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dfb6bf2e-87b1-4f97-b1e3-cead20b2d215 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd09716a091f4d98befa9909a8ad6a90'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:10.779642 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 837/1661] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:07:10 2022] DELETE /identity/v3/users/dd09716a091f4d98befa9909a8ad6a90 => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:10.783973 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31db9e-bed2-40c3-9d5b-cc6e7b9941b9 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:10.784456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31db9e-bed2-40c3-9d5b-cc6e7b9941b9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:10.785065 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31db9e-bed2-40c3-9d5b-cc6e7b9941b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:10.806482 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 825/1662] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:07:10 2022] POST /identity/v3/users => generated 290 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:10.811119 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ac025-b823-47a9-ad51-049f8275ef36 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:10.813725 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-dd5ac025-b823-47a9-ad51-049f8275ef36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:10.837290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-6a31db9e-bed2-40c3-9d5b-cc6e7b9941b9 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:10.839660 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6a31db9e-bed2-40c3-9d5b-cc6e7b9941b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:10.885535 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ac025-b823-47a9-ad51-049f8275ef36 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:11.014147 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 838/1663] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:11.073227 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbdfe2f-3618-49fe-a040-4c4acd4ab782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:11.073950 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbdfe2f-3618-49fe-a040-4c4acd4ab782 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:11.074255 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbdfe2f-3618-49fe-a040-4c4acd4ab782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:11.074509 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbdfe2f-3618-49fe-a040-4c4acd4ab782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:11.075415 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbdfe2f-3618-49fe-a040-4c4acd4ab782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:11.090476 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbdfe2f-3618-49fe-a040-4c4acd4ab782 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:11.099364 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31db9e-bed2-40c3-9d5b-cc6e7b9941b9 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:11.103360 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6a31db9e-bed2-40c3-9d5b-cc6e7b9941b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:11.136641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 826/1664] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:10 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:11.137804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbdfe2f-3618-49fe-a040-4c4acd4ab782 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:11.163027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbdfe2f-3618-49fe-a040-4c4acd4ab782 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:11.168065 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af8536-00b6-47c6-9802-019f81e599dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:11.168767 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af8536-00b6-47c6-9802-019f81e599dc admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:11.169226 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af8536-00b6-47c6-9802-019f81e599dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:11.169510 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af8536-00b6-47c6-9802-019f81e599dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:11.170863 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af8536-00b6-47c6-9802-019f81e599dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:11.173816 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af8536-00b6-47c6-9802-019f81e599dc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:11.205350 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 827/1665] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:11.206538 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af8536-00b6-47c6-9802-019f81e599dc admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:11.229804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af8536-00b6-47c6-9802-019f81e599dc admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:11.250164 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31db9e-bed2-40c3-9d5b-cc6e7b9941b9 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:11.253911 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b8d83-6806-4f7c-bc41-832678cb8ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:11.254825 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b8d83-6806-4f7c-bc41-832678cb8ce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:11.255108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b8d83-6806-4f7c-bc41-832678cb8ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:11.255386 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b8d83-6806-4f7c-bc41-832678cb8ce7 admin admin] PATH_INFO: `/v3/projects/08c9cba6de694be497659409d33421ba/users/556b8221f44a4db2a0b60225003dfdc9/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:11.256071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b7ebb0-9a00-490b-a71b-daf02798c5f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:11.257776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7ebb0-9a00-490b-a71b-daf02798c5f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:11.258438 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7ebb0-9a00-490b-a71b-daf02798c5f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:11.259024 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b7ebb0-9a00-490b-a71b-daf02798c5f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:11.278195 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b7ebb0-9a00-490b-a71b-daf02798c5f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:11.282689 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b7ebb0-9a00-490b-a71b-daf02798c5f0 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:11.296021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b8d83-6806-4f7c-bc41-832678cb8ce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c9cba6de694be497659409d33421ba, user_id=556b8221f44a4db2a0b60225003dfdc9, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:11.298743 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 839/1666] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:11.298902 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b8d83-6806-4f7c-bc41-832678cb8ce7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:11.400348 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 828/1667] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:11 2022] PUT /identity/v3/projects/08c9cba6de694be497659409d33421ba/users/556b8221f44a4db2a0b60225003dfdc9/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:11.402001 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f27ba-36bb-440a-a041-3b6b2f5e3fc9 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:11.402404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f27ba-36bb-440a-a041-3b6b2f5e3fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:11.402601 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f27ba-36bb-440a-a041-3b6b2f5e3fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:11.469475 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-6c1f27ba-36bb-440a-a041-3b6b2f5e3fc9 None None] MFA Rules not processed for user `556b8221f44a4db2a0b60225003dfdc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:11.471015 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6c1f27ba-36bb-440a-a041-3b6b2f5e3fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:11.627649 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 840/1668] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:11.632409 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b8d83-6806-4f7c-bc41-832678cb8ce7 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:11.695684 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:11.783363 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b8d83-6806-4f7c-bc41-832678cb8ce7 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:11.799545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36877e-ce3d-40ff-b8ed-5e1bbeb198ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:11.800902 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36877e-ce3d-40ff-b8ed-5e1bbeb198ee admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:11.801184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36877e-ce3d-40ff-b8ed-5e1bbeb198ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:11.801626 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36877e-ce3d-40ff-b8ed-5e1bbeb198ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:11.802466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf36877e-ce3d-40ff-b8ed-5e1bbeb198ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:11.806025 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf36877e-ce3d-40ff-b8ed-5e1bbeb198ee admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:12.005341 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 829/1669] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:07:11 2022] POST /identity/v3/projects => generated 381 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:12.007688 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f27ba-36bb-440a-a041-3b6b2f5e3fc9 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:12.027461 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f27ba-36bb-440a-a041-3b6b2f5e3fc9 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:12.213729 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:12.217842 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3b8bc-cd4f-4582-a9ba-2492929ad4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:12.221515 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3b8bc-cd4f-4582-a9ba-2492929ad4b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:12.221515 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3b8bc-cd4f-4582-a9ba-2492929ad4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:12.221515 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3b8bc-cd4f-4582-a9ba-2492929ad4b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:12.221515 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3b8bc-cd4f-4582-a9ba-2492929ad4b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:12.224514 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3b8bc-cd4f-4582-a9ba-2492929ad4b6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:12.377456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 841/1670] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:07:12 2022] POST /identity/v3/users => generated 297 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:12.379123 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36877e-ce3d-40ff-b8ed-5e1bbeb198ee admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:12.442190 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36877e-ce3d-40ff-b8ed-5e1bbeb198ee admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:12.470710 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7fc5f3-048b-431b-93ec-4cf5b2ff5796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:12.472541 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7fc5f3-048b-431b-93ec-4cf5b2ff5796 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:12.472928 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7fc5f3-048b-431b-93ec-4cf5b2ff5796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:12.473541 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7fc5f3-048b-431b-93ec-4cf5b2ff5796 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:12.474757 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7fc5f3-048b-431b-93ec-4cf5b2ff5796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:12.477331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7fc5f3-048b-431b-93ec-4cf5b2ff5796 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:12.500269 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 830/1671] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:12.507864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3b8bc-cd4f-4582-a9ba-2492929ad4b6 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:12.529559 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3b8bc-cd4f-4582-a9ba-2492929ad4b6 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:12.551311 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad32713-97ec-4dca-8b14-0d46eb89a2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:12.552515 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad32713-97ec-4dca-8b14-0d46eb89a2c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:12.553097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad32713-97ec-4dca-8b14-0d46eb89a2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:12.553518 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad32713-97ec-4dca-8b14-0d46eb89a2c6 admin admin] PATH_INFO: `/v3/projects/4cd09a6fd35d41c3997c91c147da928d/users/8f4ead08228347f89e7e8b2e427de608/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:12.612346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad32713-97ec-4dca-8b14-0d46eb89a2c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cd09a6fd35d41c3997c91c147da928d, user_id=8f4ead08228347f89e7e8b2e427de608, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:12.618656 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad32713-97ec-4dca-8b14-0d46eb89a2c6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:12.712845 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 842/1672] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:12 2022] PUT /identity/v3/projects/4cd09a6fd35d41c3997c91c147da928d/users/8f4ead08228347f89e7e8b2e427de608/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:12.713381 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7fc5f3-048b-431b-93ec-4cf5b2ff5796 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:12.742949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7fc5f3-048b-431b-93ec-4cf5b2ff5796 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:12.937237 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd3ab2-398a-4cd9-879f-3ba31efa809f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:12.938900 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd3ab2-398a-4cd9-879f-3ba31efa809f admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:12.939243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd3ab2-398a-4cd9-879f-3ba31efa809f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:12.939523 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd3ab2-398a-4cd9-879f-3ba31efa809f admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:12.940438 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febd3ab2-398a-4cd9-879f-3ba31efa809f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:12.942705 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febd3ab2-398a-4cd9-879f-3ba31efa809f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:12.959422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 831/1673] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:12.970616 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad32713-97ec-4dca-8b14-0d46eb89a2c6 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:13.003750 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9acde5-57d5-4681-bc5e-d1da4008c13f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:13.003750 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9acde5-57d5-4681-bc5e-d1da4008c13f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:13.004101 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9acde5-57d5-4681-bc5e-d1da4008c13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:13.011677 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad32713-97ec-4dca-8b14-0d46eb89a2c6 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:13.032155 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5be714-7dc7-42ad-aa0e-7dd400cbafff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:13.032155 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5be714-7dc7-42ad-aa0e-7dd400cbafff admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:13.032436 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5be714-7dc7-42ad-aa0e-7dd400cbafff admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:13.032519 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5be714-7dc7-42ad-aa0e-7dd400cbafff admin admin] PATH_INFO: `/v3/projects/4cd09a6fd35d41c3997c91c147da928d/users/8f4ead08228347f89e7e8b2e427de608/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:13.041517 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5be714-7dc7-42ad-aa0e-7dd400cbafff admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cd09a6fd35d41c3997c91c147da928d, user_id=8f4ead08228347f89e7e8b2e427de608, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:13.049576 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5be714-7dc7-42ad-aa0e-7dd400cbafff admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:13.086913 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 843/1674] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:12 2022] PUT /identity/v3/projects/4cd09a6fd35d41c3997c91c147da928d/users/8f4ead08228347f89e7e8b2e427de608/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:13.099169 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb1354-2300-427e-8426-20aa7bfcde19 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:13.099497 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb1354-2300-427e-8426-20aa7bfcde19 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:13.099782 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb1354-2300-427e-8426-20aa7bfcde19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:13.162019 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7e9acde5-57d5-4681-bc5e-d1da4008c13f None None] MFA Rules not processed for user `8a5b19c8c0d44e1f93bb1dea72884cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:13.166347 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7e9acde5-57d5-4681-bc5e-d1da4008c13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:13.179714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f8fb1354-2300-427e-8426-20aa7bfcde19 None None] MFA Rules not processed for user `8f4ead08228347f89e7e8b2e427de608`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:13.187872 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f8fb1354-2300-427e-8426-20aa7bfcde19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:13.399863 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 832/1675] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:13.407764 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9acde5-57d5-4681-bc5e-d1da4008c13f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:13.409123 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7e9acde5-57d5-4681-bc5e-d1da4008c13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:13.422109 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 844/1676] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:13.435925 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb1354-2300-427e-8426-20aa7bfcde19 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:13.439374 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f8fb1354-2300-427e-8426-20aa7bfcde19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:13.538357 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:13.630322 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb1354-2300-427e-8426-20aa7bfcde19 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:13.641961 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e34e80-eb12-47d1-a477-d51052ef60fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:13.642680 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e34e80-eb12-47d1-a477-d51052ef60fe service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:13.642965 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e34e80-eb12-47d1-a477-d51052ef60fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:13.643242 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e34e80-eb12-47d1-a477-d51052ef60fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:13.701164 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e34e80-eb12-47d1-a477-d51052ef60fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:13.702696 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e34e80-eb12-47d1-a477-d51052ef60fe service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:13.708261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9acde5-57d5-4681-bc5e-d1da4008c13f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:13.725915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7179bb78-3264-4223-807b-cbfa63f5c55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:13.727620 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179bb78-3264-4223-807b-cbfa63f5c55c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:13.728424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179bb78-3264-4223-807b-cbfa63f5c55c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:13.730483 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179bb78-3264-4223-807b-cbfa63f5c55c admin admin] PATH_INFO: `/v3/users/bb1d18a000dd48d1b50569857c1783c4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:13.733481 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7179bb78-3264-4223-807b-cbfa63f5c55c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb1d18a000dd48d1b50569857c1783c4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:13.742451 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7179bb78-3264-4223-807b-cbfa63f5c55c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:13.743442 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 845/1677] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:13.745094 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0db65-2239-4cb2-a7b1-c5e099eaa94f None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:13.745391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0db65-2239-4cb2-a7b1-c5e099eaa94f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:13.745678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0db65-2239-4cb2-a7b1-c5e099eaa94f None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:13.762059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 846/1678] 10.222.0.44 () {54 vars in 961 bytes} [Thu Jun 23 20:07:13 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:07:13.763063 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f0db65-2239-4cb2-a7b1-c5e099eaa94f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:13.767511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-87f0db65-2239-4cb2-a7b1-c5e099eaa94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:13.815996 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:13.817079 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f0db65-2239-4cb2-a7b1-c5e099eaa94f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:13.829846 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3feb17c-725c-4681-b8a2-aaaf891f9b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:13.830778 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3feb17c-725c-4681-b8a2-aaaf891f9b87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:13.831056 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3feb17c-725c-4681-b8a2-aaaf891f9b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:13.831995 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3feb17c-725c-4681-b8a2-aaaf891f9b87 admin admin] PATH_INFO: `/v3/projects/8c55f3d14ad14df5a59d0d43a21c7ed9` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:13.835659 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3feb17c-725c-4681-b8a2-aaaf891f9b87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c55f3d14ad14df5a59d0d43a21c7ed9)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:13.837663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3feb17c-725c-4681-b8a2-aaaf891f9b87 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:13.990668 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7179bb78-3264-4223-807b-cbfa63f5c55c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb1d18a000dd48d1b50569857c1783c4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:13.994808 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7179bb78-3264-4223-807b-cbfa63f5c55c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb1d18a000dd48d1b50569857c1783c4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:14.027384 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d3feb17c-725c-4681-b8a2-aaaf891f9b87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c55f3d14ad14df5a59d0d43a21c7ed9', 'request_id': 'req-d3feb17c-725c-4681-b8a2-aaaf891f9b87'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:14.046651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7179bb78-3264-4223-807b-cbfa63f5c55c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb1d18a000dd48d1b50569857c1783c4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:14.063150 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7179bb78-3264-4223-807b-cbfa63f5c55c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb1d18a000dd48d1b50569857c1783c4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:14.142640 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d3feb17c-725c-4681-b8a2-aaaf891f9b87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c55f3d14ad14df5a59d0d43a21c7ed9', 'request_id': 'req-d3feb17c-725c-4681-b8a2-aaaf891f9b87'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:14.161144 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d3feb17c-725c-4681-b8a2-aaaf891f9b87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c55f3d14ad14df5a59d0d43a21c7ed9', 'request_id': 'req-d3feb17c-725c-4681-b8a2-aaaf891f9b87'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:14.191525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7179bb78-3264-4223-807b-cbfa63f5c55c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb1d18a000dd48d1b50569857c1783c4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:14.195653 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d3feb17c-725c-4681-b8a2-aaaf891f9b87 admin admin] The token cache is being invalidate because project 8c55f3d14ad14df5a59d0d43a21c7ed9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:07:14.195950 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d3feb17c-725c-4681-b8a2-aaaf891f9b87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:14.199291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d3feb17c-725c-4681-b8a2-aaaf891f9b87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:14.200677 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 833/1679] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:07:13 2022] DELETE /identity/v3/users/bb1d18a000dd48d1b50569857c1783c4 => generated 0 bytes in 800 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:14.201557 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 847/1680] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:07:13 2022] DELETE /identity/v3/projects/8c55f3d14ad14df5a59d0d43a21c7ed9 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:14.203239 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae0e17-4f49-439d-9305-6791a9f22cdd None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:14.205144 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ed5e4-700b-4160-acb4-433d63117c70 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:14.205608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ed5e4-700b-4160-acb4-433d63117c70 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:14.207081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ed5e4-700b-4160-acb4-433d63117c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:14.207588 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae0e17-4f49-439d-9305-6791a9f22cdd None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:14.207952 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae0e17-4f49-439d-9305-6791a9f22cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:14.251150 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-774ed5e4-700b-4160-acb4-433d63117c70 None None] MFA Rules not processed for user `a63d1d902c024c97a38f7d92f73814d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:14.252426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-774ed5e4-700b-4160-acb4-433d63117c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:14.274814 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-03ae0e17-4f49-439d-9305-6791a9f22cdd None None] MFA Rules not processed for user `f041449191c94a5ca5a38c29dd146a15`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:14.291650 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-03ae0e17-4f49-439d-9305-6791a9f22cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:14.453505 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 848/1681] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:14.454918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4ec5e-9c16-48bc-97f4-0d423de46af1 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:14.455207 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4ec5e-9c16-48bc-97f4-0d423de46af1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:14.455487 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4ec5e-9c16-48bc-97f4-0d423de46af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:14.484491 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 834/1682] 10.222.0.44 () {58 vars in 1058 bytes} [Thu Jun 23 20:07:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:14.486900 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ed5e4-700b-4160-acb4-433d63117c70 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:14.491636 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-774ed5e4-700b-4160-acb4-433d63117c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:14.515733 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-31f4ec5e-9c16-48bc-97f4-0d423de46af1 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:14.517014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-31f4ec5e-9c16-48bc-97f4-0d423de46af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:14.661225 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 849/1683] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:14.663753 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff334912-24ba-436f-9392-7f1e544f0d7c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:14.672581 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff334912-24ba-436f-9392-7f1e544f0d7c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:14.673106 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff334912-24ba-436f-9392-7f1e544f0d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:14.718739 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ff334912-24ba-436f-9392-7f1e544f0d7c None None] MFA Rules not processed for user `f041449191c94a5ca5a38c29dd146a15`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:14.720097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ff334912-24ba-436f-9392-7f1e544f0d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:14.722942 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ed5e4-700b-4160-acb4-433d63117c70 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:14.735253 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 850/1684] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:14.736544 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff334912-24ba-436f-9392-7f1e544f0d7c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:14.742737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5721e3-3cbc-4894-b790-b78db6f95387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:14.745925 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5721e3-3cbc-4894-b790-b78db6f95387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:14.746970 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5721e3-3cbc-4894-b790-b78db6f95387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:14.748787 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5721e3-3cbc-4894-b790-b78db6f95387 admin admin] PATH_INFO: `/v3/users/2066ab7ad8e04aac83b18b50c79a09c3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:14.754486 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5721e3-3cbc-4894-b790-b78db6f95387 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2066ab7ad8e04aac83b18b50c79a09c3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:14.760033 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5721e3-3cbc-4894-b790-b78db6f95387 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:14.819193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff334912-24ba-436f-9392-7f1e544f0d7c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:14.842960 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985d362-49c9-4b23-b680-eccde00b8404 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:14.843769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985d362-49c9-4b23-b680-eccde00b8404 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:14.845328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985d362-49c9-4b23-b680-eccde00b8404 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:14.851389 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985d362-49c9-4b23-b680-eccde00b8404 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:14.853312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6985d362-49c9-4b23-b680-eccde00b8404 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:14.989146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6985d362-49c9-4b23-b680-eccde00b8404 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:14.991003 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6985d362-49c9-4b23-b680-eccde00b8404 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:15.002321 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 851/1685] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:15.004070 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985d362-49c9-4b23-b680-eccde00b8404 service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:15.006129 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6985d362-49c9-4b23-b680-eccde00b8404 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:15.204629 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-bf5721e3-3cbc-4894-b790-b78db6f95387 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2066ab7ad8e04aac83b18b50c79a09c3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:15.207457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-bf5721e3-3cbc-4894-b790-b78db6f95387 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2066ab7ad8e04aac83b18b50c79a09c3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:15.255778 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985d362-49c9-4b23-b680-eccde00b8404 service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:15.263393 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-bf5721e3-3cbc-4894-b790-b78db6f95387 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2066ab7ad8e04aac83b18b50c79a09c3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:15.286171 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-bf5721e3-3cbc-4894-b790-b78db6f95387 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2066ab7ad8e04aac83b18b50c79a09c3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:15.321387 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe9b726-7e2a-4a9a-bc92-4d9a11f25281 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:15.322388 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9b726-7e2a-4a9a-bc92-4d9a11f25281 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:15.322681 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9b726-7e2a-4a9a-bc92-4d9a11f25281 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:15.322953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9b726-7e2a-4a9a-bc92-4d9a11f25281 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:15.342708 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe9b726-7e2a-4a9a-bc92-4d9a11f25281 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:15.344758 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe9b726-7e2a-4a9a-bc92-4d9a11f25281 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:15.362437 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 852/1686] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:15.364767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f54149-e7fe-4922-8db5-bd9b23d71a24 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:15.365275 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f54149-e7fe-4922-8db5-bd9b23d71a24 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:15.365761 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f54149-e7fe-4922-8db5-bd9b23d71a24 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:15.367958 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 853/1687] 10.222.0.44 () {54 vars in 961 bytes} [Thu Jun 23 20:07:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:07:15.369439 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f54149-e7fe-4922-8db5-bd9b23d71a24 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:15.375599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f3f54149-e7fe-4922-8db5-bd9b23d71a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:15.385417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:15.391053 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-bf5721e3-3cbc-4894-b790-b78db6f95387 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2066ab7ad8e04aac83b18b50c79a09c3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:15.409354 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 835/1688] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:07:14 2022] DELETE /identity/v3/users/2066ab7ad8e04aac83b18b50c79a09c3 => generated 0 bytes in 924 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:15.416274 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e3489-2a2d-4f0a-b864-e1af40b35086 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:15.418243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e3489-2a2d-4f0a-b864-e1af40b35086 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:15.419358 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e3489-2a2d-4f0a-b864-e1af40b35086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:15.464809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8f6e3489-2a2d-4f0a-b864-e1af40b35086 None None] MFA Rules not processed for user `a63d1d902c024c97a38f7d92f73814d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:15.466278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8f6e3489-2a2d-4f0a-b864-e1af40b35086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:15.546541 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:15.577549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f54149-e7fe-4922-8db5-bd9b23d71a24 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:15.599272 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 836/1689] 10.222.0.44 () {58 vars in 1058 bytes} [Thu Jun 23 20:07:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:15.606787 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e3489-2a2d-4f0a-b864-e1af40b35086 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:15.632481 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6e3489-2a2d-4f0a-b864-e1af40b35086 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:15.650074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558709e8-4f0b-4b36-86ea-ef769a985f92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:15.650787 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558709e8-4f0b-4b36-86ea-ef769a985f92 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:15.651298 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558709e8-4f0b-4b36-86ea-ef769a985f92 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:15.651581 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558709e8-4f0b-4b36-86ea-ef769a985f92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:15.653684 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-558709e8-4f0b-4b36-86ea-ef769a985f92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:15.705896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69262951-e385-46a9-84d0-99fbf9198a8c tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4ead08228347f89e7e8b2e427de608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cd09a6fd35d41c3997c91c147da928d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:15.706821 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69262951-e385-46a9-84d0-99fbf9198a8c tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:15.706940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69262951-e385-46a9-84d0-99fbf9198a8c tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:15.707135 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69262951-e385-46a9-84d0-99fbf9198a8c tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:15.707892 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69262951-e385-46a9-84d0-99fbf9198a8c tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:07:15.708233 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69262951-e385-46a9-84d0-99fbf9198a8c tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:15.710010 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69262951-e385-46a9-84d0-99fbf9198a8c tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:15.746801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 854/1690] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:07:15.747855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69262951-e385-46a9-84d0-99fbf9198a8c tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:15.749665 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-69262951-e385-46a9-84d0-99fbf9198a8c tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:15.772097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69262951-e385-46a9-84d0-99fbf9198a8c tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:15.794253 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75169625-12eb-4654-ac2c-762e539778d8 tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4ead08228347f89e7e8b2e427de608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cd09a6fd35d41c3997c91c147da928d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:15.794950 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75169625-12eb-4654-ac2c-762e539778d8 tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:15.796837 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75169625-12eb-4654-ac2c-762e539778d8 tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:15.797132 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75169625-12eb-4654-ac2c-762e539778d8 tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:15.797859 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75169625-12eb-4654-ac2c-762e539778d8 tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:15.813577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75169625-12eb-4654-ac2c-762e539778d8 tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:15.935158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558709e8-4f0b-4b36-86ea-ef769a985f92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:15.936850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558709e8-4f0b-4b36-86ea-ef769a985f92 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:15.984978 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 855/1691] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:07:15 2022] POST /identity/v3/projects => generated 354 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:07:15.984978 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad8e18-8db5-478b-ab79-cb061e397854 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:15.986609 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad8e18-8db5-478b-ab79-cb061e397854 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:15.990975 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad8e18-8db5-478b-ab79-cb061e397854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:15.993241 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 837/1692] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:16.015578 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558709e8-4f0b-4b36-86ea-ef769a985f92 service nova] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:16.038685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558709e8-4f0b-4b36-86ea-ef769a985f92 service nova] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:16.090378 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-caad8e18-8db5-478b-ab79-cb061e397854 None None] MFA Rules not processed for user `a3be90df779b4ddb8639d825fcb41234`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:16.090378 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-caad8e18-8db5-478b-ab79-cb061e397854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:16.254217 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c37de26-6084-423b-abdd-b70062a3ad03 tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4ead08228347f89e7e8b2e427de608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cd09a6fd35d41c3997c91c147da928d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:16.255910 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37de26-6084-423b-abdd-b70062a3ad03 tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:16.256472 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37de26-6084-423b-abdd-b70062a3ad03 tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:16.258364 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c37de26-6084-423b-abdd-b70062a3ad03 tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:16.259692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c37de26-6084-423b-abdd-b70062a3ad03 tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:07:16.260654 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c37de26-6084-423b-abdd-b70062a3ad03 tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:16.262520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c37de26-6084-423b-abdd-b70062a3ad03 tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:16.286550 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 838/1693] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:07:16.330715 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 856/1694] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:16.347791 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c37de26-6084-423b-abdd-b70062a3ad03 tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:16.350336 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8c37de26-6084-423b-abdd-b70062a3ad03 tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:16.403973 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad8e18-8db5-478b-ab79-cb061e397854 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:16.437184 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caad8e18-8db5-478b-ab79-cb061e397854 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:16.446913 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98225168-441e-44ae-9688-79764c053a42 tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8f4ead08228347f89e7e8b2e427de608', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4cd09a6fd35d41c3997c91c147da928d', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:16.449026 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98225168-441e-44ae-9688-79764c053a42 tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:16.449026 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98225168-441e-44ae-9688-79764c053a42 tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:16.449026 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98225168-441e-44ae-9688-79764c053a42 tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] PATH_INFO: `/v3/projects/481a83cc6e594be3a07a9857ed3121bb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:16.449955 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98225168-441e-44ae-9688-79764c053a42 tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] RBAC: Authorizing `identity:delete_project(project_id=481a83cc6e594be3a07a9857ed3121bb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:16.452422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98225168-441e-44ae-9688-79764c053a42 tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:16.553276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c37de26-6084-423b-abdd-b70062a3ad03 tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:16.564009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7fa1f7-dce2-44d7-ac41-51c3a52bd6a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:16.564732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fa1f7-dce2-44d7-ac41-51c3a52bd6a7 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:16.566115 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fa1f7-dce2-44d7-ac41-51c3a52bd6a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:16.566115 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fa1f7-dce2-44d7-ac41-51c3a52bd6a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:16.598108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7fa1f7-dce2-44d7-ac41-51c3a52bd6a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:16.599757 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7fa1f7-dce2-44d7-ac41-51c3a52bd6a7 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:16.615658 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 839/1695] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:16.617478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bc546-1a63-4e6f-a432-710c17c7d4e1 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:16.617478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bc546-1a63-4e6f-a432-710c17c7d4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:16.617769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bc546-1a63-4e6f-a432-710c17c7d4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:16.687603 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-541bc546-1a63-4e6f-a432-710c17c7d4e1 None None] MFA Rules not processed for user `5645338e8194426eac77359c0cb233f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:16.689148 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-541bc546-1a63-4e6f-a432-710c17c7d4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:16.710539 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-98225168-441e-44ae-9688-79764c053a42 tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '481a83cc6e594be3a07a9857ed3121bb', 'request_id': 'req-98225168-441e-44ae-9688-79764c053a42'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:16.801940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-98225168-441e-44ae-9688-79764c053a42 tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '481a83cc6e594be3a07a9857ed3121bb', 'request_id': 'req-98225168-441e-44ae-9688-79764c053a42'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:16.809728 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-98225168-441e-44ae-9688-79764c053a42 tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '481a83cc6e594be3a07a9857ed3121bb', 'request_id': 'req-98225168-441e-44ae-9688-79764c053a42'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:16.846866 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-98225168-441e-44ae-9688-79764c053a42 tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] The token cache is being invalidate because project 481a83cc6e594be3a07a9857ed3121bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:07:16.846866 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-98225168-441e-44ae-9688-79764c053a42 tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:16.846866 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-98225168-441e-44ae-9688-79764c053a42 tempest-VolumeQuotaClassesTest-1143009620 tempest-VolumeQuotaClassesTest-1143009620-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:16.853462 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 857/1696] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:07:16 2022] DELETE /identity/v3/projects/481a83cc6e594be3a07a9857ed3121bb => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:16.854993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8a759-9763-4eee-bce6-8cdaaf26dbbc None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:16.855328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8a759-9763-4eee-bce6-8cdaaf26dbbc None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:16.855556 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8a759-9763-4eee-bce6-8cdaaf26dbbc None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:16.858246 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 858/1697] 10.222.0.44 () {54 vars in 912 bytes} [Thu Jun 23 20:07:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:07:16.864258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60c180-a696-4229-8ce7-bcd09f26ac01 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:16.868178 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60c180-a696-4229-8ce7-bcd09f26ac01 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:16.869519 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60c180-a696-4229-8ce7-bcd09f26ac01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:16.876732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 840/1698] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:16.893163 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38454ffe-eaa2-4904-af96-990b6d3db9b6 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:16.893163 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38454ffe-eaa2-4904-af96-990b6d3db9b6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:16.893163 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38454ffe-eaa2-4904-af96-990b6d3db9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:16.942144 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-cf60c180-a696-4229-8ce7-bcd09f26ac01 None None] MFA Rules not processed for user `a63d1d902c024c97a38f7d92f73814d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:16.946169 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-cf60c180-a696-4229-8ce7-bcd09f26ac01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:16.949868 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-38454ffe-eaa2-4904-af96-990b6d3db9b6 None None] MFA Rules not processed for user `5645338e8194426eac77359c0cb233f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:16.951728 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-38454ffe-eaa2-4904-af96-990b6d3db9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:16.978921 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 841/1699] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:16.989701 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38454ffe-eaa2-4904-af96-990b6d3db9b6 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:16.991925 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-38454ffe-eaa2-4904-af96-990b6d3db9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:17.092280 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 859/1700] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jun 23 20:07:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:17.105288 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60c180-a696-4229-8ce7-bcd09f26ac01 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:17.107784 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-cf60c180-a696-4229-8ce7-bcd09f26ac01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:17.187900 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38454ffe-eaa2-4904-af96-990b6d3db9b6 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:17.212981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f154a0cc-f472-47c4-b8a0-773be97a20e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:17.212981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154a0cc-f472-47c4-b8a0-773be97a20e2 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:17.212981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154a0cc-f472-47c4-b8a0-773be97a20e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:17.212981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154a0cc-f472-47c4-b8a0-773be97a20e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:17.218173 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f154a0cc-f472-47c4-b8a0-773be97a20e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:17.262386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60c180-a696-4229-8ce7-bcd09f26ac01 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:17.267527 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0638b1d-e596-46c9-96f4-0c7a60aa4cf5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd72f952df34ad8853fe84673a41f45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:17.269237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0638b1d-e596-46c9-96f4-0c7a60aa4cf5 service glance] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:17.269377 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f154a0cc-f472-47c4-b8a0-773be97a20e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:17.270256 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0638b1d-e596-46c9-96f4-0c7a60aa4cf5 service glance] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:17.270677 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0638b1d-e596-46c9-96f4-0c7a60aa4cf5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:17.272233 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f154a0cc-f472-47c4-b8a0-773be97a20e2 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:17.289064 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0638b1d-e596-46c9-96f4-0c7a60aa4cf5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:17.301328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0638b1d-e596-46c9-96f4-0c7a60aa4cf5 service glance] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:17.334040 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 842/1701] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:16 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 20:07:17.368103 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f154a0cc-f472-47c4-b8a0-773be97a20e2 service nova] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:17.368103 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f154a0cc-f472-47c4-b8a0-773be97a20e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:17.375528 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 860/1702] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 20:07:17.424043 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0638b1d-e596-46c9-96f4-0c7a60aa4cf5 service glance] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:17.426193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a0638b1d-e596-46c9-96f4-0c7a60aa4cf5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:17.494535 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f154a0cc-f472-47c4-b8a0-773be97a20e2 service nova] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:17.511176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e251023-60aa-427f-9bce-4b9399a23622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:17.512537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e251023-60aa-427f-9bce-4b9399a23622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:17.512537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e251023-60aa-427f-9bce-4b9399a23622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:17.512720 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e251023-60aa-427f-9bce-4b9399a23622 admin admin] PATH_INFO: `/v3/projects/e208585b066d4699b288644e48300e7f` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:17.516717 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e251023-60aa-427f-9bce-4b9399a23622 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e208585b066d4699b288644e48300e7f)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:17.520425 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e251023-60aa-427f-9bce-4b9399a23622 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:17.588490 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0638b1d-e596-46c9-96f4-0c7a60aa4cf5 service glance] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:17.612342 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac848fb-a558-4cd5-8fbf-f1d205a97371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:17.612639 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac848fb-a558-4cd5-8fbf-f1d205a97371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:17.615054 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac848fb-a558-4cd5-8fbf-f1d205a97371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:17.616498 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac848fb-a558-4cd5-8fbf-f1d205a97371 admin admin] PATH_INFO: `/v3/projects/e9733cc852d54b15a7fab334a1195114` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:17.621581 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac848fb-a558-4cd5-8fbf-f1d205a97371 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9733cc852d54b15a7fab334a1195114)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:17.634827 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac848fb-a558-4cd5-8fbf-f1d205a97371 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:17.819583 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4e251023-60aa-427f-9bce-4b9399a23622 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e208585b066d4699b288644e48300e7f', 'request_id': 'req-4e251023-60aa-427f-9bce-4b9399a23622'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:17.848377 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4e251023-60aa-427f-9bce-4b9399a23622 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e208585b066d4699b288644e48300e7f', 'request_id': 'req-4e251023-60aa-427f-9bce-4b9399a23622'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:17.863267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4e251023-60aa-427f-9bce-4b9399a23622 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e208585b066d4699b288644e48300e7f', 'request_id': 'req-4e251023-60aa-427f-9bce-4b9399a23622'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:17.904244 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0ac848fb-a558-4cd5-8fbf-f1d205a97371 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9733cc852d54b15a7fab334a1195114', 'request_id': 'req-0ac848fb-a558-4cd5-8fbf-f1d205a97371'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:17.996077 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4e251023-60aa-427f-9bce-4b9399a23622 admin admin] The token cache is being invalidate because project e208585b066d4699b288644e48300e7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:07:17.996655 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4e251023-60aa-427f-9bce-4b9399a23622 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:17.997616 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4e251023-60aa-427f-9bce-4b9399a23622 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:17.999788 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 843/1703] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:07:17 2022] DELETE /identity/v3/projects/e208585b066d4699b288644e48300e7f => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:07:18.004394 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216efe2-ab80-4405-a313-2513c450e71f None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:18.008175 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216efe2-ab80-4405-a313-2513c450e71f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:18.018400 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216efe2-ab80-4405-a313-2513c450e71f None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:18.024184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 844/1704] 10.222.0.44 () {54 vars in 912 bytes} [Thu Jun 23 20:07:18 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:07:18.033361 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf9180-57de-4f68-bec7-933b2ef7f90c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:18.034118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf9180-57de-4f68-bec7-933b2ef7f90c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:18.035391 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf9180-57de-4f68-bec7-933b2ef7f90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:18.081281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-dbbf9180-57de-4f68-bec7-933b2ef7f90c None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:18.086822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-dbbf9180-57de-4f68-bec7-933b2ef7f90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:18.141643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0ac848fb-a558-4cd5-8fbf-f1d205a97371 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9733cc852d54b15a7fab334a1195114', 'request_id': 'req-0ac848fb-a558-4cd5-8fbf-f1d205a97371'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:18.157874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0ac848fb-a558-4cd5-8fbf-f1d205a97371 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9733cc852d54b15a7fab334a1195114', 'request_id': 'req-0ac848fb-a558-4cd5-8fbf-f1d205a97371'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:18.182747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0ac848fb-a558-4cd5-8fbf-f1d205a97371 admin admin] The token cache is being invalidate because project e9733cc852d54b15a7fab334a1195114 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:07:18.185260 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0ac848fb-a558-4cd5-8fbf-f1d205a97371 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:18.185260 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0ac848fb-a558-4cd5-8fbf-f1d205a97371 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:18.191934 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 861/1705] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:07:17 2022] DELETE /identity/v3/projects/e9733cc852d54b15a7fab334a1195114 => generated 0 bytes in 765 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:18.194673 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13af10-6b85-4e10-b1ca-1d547ec03d5a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:18.198617 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13af10-6b85-4e10-b1ca-1d547ec03d5a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:18.199161 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13af10-6b85-4e10-b1ca-1d547ec03d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:18.255992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-bd13af10-6b85-4e10-b1ca-1d547ec03d5a None None] MFA Rules not processed for user `a63d1d902c024c97a38f7d92f73814d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:18.258016 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 845/1706] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:18.258172 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bd13af10-6b85-4e10-b1ca-1d547ec03d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:18.262010 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf9180-57de-4f68-bec7-933b2ef7f90c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:18.271067 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-dbbf9180-57de-4f68-bec7-933b2ef7f90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:18.293689 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf9180-57de-4f68-bec7-933b2ef7f90c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:18.300174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343e50e3-d1a3-48e8-9820-3e033ac9fc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:18.301712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e50e3-d1a3-48e8-9820-3e033ac9fc1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:18.301999 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e50e3-d1a3-48e8-9820-3e033ac9fc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:18.302281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e50e3-d1a3-48e8-9820-3e033ac9fc1d admin admin] PATH_INFO: `/v3/users/24bcfd9934b049999addbd0246cf03ed` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:18.303565 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343e50e3-d1a3-48e8-9820-3e033ac9fc1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=24bcfd9934b049999addbd0246cf03ed)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:18.309613 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343e50e3-d1a3-48e8-9820-3e033ac9fc1d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:18.384389 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 862/1707] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jun 23 20:07:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:18.386503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae3a92-6110-4827-8a22-72dbf8e4a4ee None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:18.386807 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae3a92-6110-4827-8a22-72dbf8e4a4ee None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:18.387091 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae3a92-6110-4827-8a22-72dbf8e4a4ee None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:18.393119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 863/1708] 10.222.0.44 () {54 vars in 912 bytes} [Thu Jun 23 20:07:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:07:18.395162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae3a92-6110-4827-8a22-72dbf8e4a4ee None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:18.399669 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a0ae3a92-6110-4827-8a22-72dbf8e4a4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:18.429927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:18.440148 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae3a92-6110-4827-8a22-72dbf8e4a4ee None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:18.452462 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda77d24-759d-490b-85d6-4f688f2dd08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:18.453460 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda77d24-759d-490b-85d6-4f688f2dd08f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:18.453842 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda77d24-759d-490b-85d6-4f688f2dd08f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:18.454212 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda77d24-759d-490b-85d6-4f688f2dd08f admin admin] PATH_INFO: `/v3/users/556b8221f44a4db2a0b60225003dfdc9` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:18.457097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda77d24-759d-490b-85d6-4f688f2dd08f admin admin] RBAC: Authorizing `identity:delete_user(user_id=556b8221f44a4db2a0b60225003dfdc9)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:18.459271 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda77d24-759d-490b-85d6-4f688f2dd08f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:18.626574 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-343e50e3-d1a3-48e8-9820-3e033ac9fc1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24bcfd9934b049999addbd0246cf03ed'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:18.627672 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-343e50e3-d1a3-48e8-9820-3e033ac9fc1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24bcfd9934b049999addbd0246cf03ed'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:18.650103 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-343e50e3-d1a3-48e8-9820-3e033ac9fc1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24bcfd9934b049999addbd0246cf03ed'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:18.665801 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-343e50e3-d1a3-48e8-9820-3e033ac9fc1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24bcfd9934b049999addbd0246cf03ed'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:18.745307 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-343e50e3-d1a3-48e8-9820-3e033ac9fc1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24bcfd9934b049999addbd0246cf03ed'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:18.753584 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 846/1709] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:07:18 2022] DELETE /identity/v3/users/24bcfd9934b049999addbd0246cf03ed => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:18.756384 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457d390-eaaf-4c92-ab3d-d81f2ef240fd None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:18.756762 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457d390-eaaf-4c92-ab3d-d81f2ef240fd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:18.756981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457d390-eaaf-4c92-ab3d-d81f2ef240fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:18.762197 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dda77d24-759d-490b-85d6-4f688f2dd08f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '556b8221f44a4db2a0b60225003dfdc9'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:18.762882 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dda77d24-759d-490b-85d6-4f688f2dd08f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '556b8221f44a4db2a0b60225003dfdc9'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:18.799688 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d457d390-eaaf-4c92-ab3d-d81f2ef240fd None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:18.800984 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d457d390-eaaf-4c92-ab3d-d81f2ef240fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:18.818298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:18.859424 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dda77d24-759d-490b-85d6-4f688f2dd08f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '556b8221f44a4db2a0b60225003dfdc9'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:18.859849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dda77d24-759d-490b-85d6-4f688f2dd08f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '556b8221f44a4db2a0b60225003dfdc9'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:18.889263 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dda77d24-759d-490b-85d6-4f688f2dd08f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '556b8221f44a4db2a0b60225003dfdc9'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:18.926863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 864/1710] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:07:18 2022] DELETE /identity/v3/users/556b8221f44a4db2a0b60225003dfdc9 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:18.928098 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda77d24-759d-490b-85d6-4f688f2dd08f admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:18.934228 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-dda77d24-759d-490b-85d6-4f688f2dd08f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:18.992556 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 847/1711] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:18.998564 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e2fb8-10fb-4157-81e4-65c581498949 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:18.998876 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e2fb8-10fb-4157-81e4-65c581498949 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:18.999159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e2fb8-10fb-4157-81e4-65c581498949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:19.103484 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda77d24-759d-490b-85d6-4f688f2dd08f admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:19.108844 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac450c-a449-4395-b4f9-15f6f793f76f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd72f952df34ad8853fe84673a41f45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:19.109961 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac450c-a449-4395-b4f9-15f6f793f76f service glance] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:19.110851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac450c-a449-4395-b4f9-15f6f793f76f service glance] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:19.111619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac450c-a449-4395-b4f9-15f6f793f76f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:19.113442 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-423e2fb8-10fb-4157-81e4-65c581498949 None None] MFA Rules not processed for user `332fbbefba114da4a01fdd496fff1e31`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:19.114560 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8aac450c-a449-4395-b4f9-15f6f793f76f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:19.114821 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-423e2fb8-10fb-4157-81e4-65c581498949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:19.215933 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 848/1712] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jun 23 20:07:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:19.221189 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89db3a8-5849-4b84-9188-3d51b01d5e7b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:19.222233 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89db3a8-5849-4b84-9188-3d51b01d5e7b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:19.222712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89db3a8-5849-4b84-9188-3d51b01d5e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:19.281434 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b89db3a8-5849-4b84-9188-3d51b01d5e7b None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:19.282681 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b89db3a8-5849-4b84-9188-3d51b01d5e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:19.320706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac450c-a449-4395-b4f9-15f6f793f76f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:19.325207 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac450c-a449-4395-b4f9-15f6f793f76f service glance] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:19.342423 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 865/1713] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:19.343800 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac450c-a449-4395-b4f9-15f6f793f76f service glance] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:19.441118 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac450c-a449-4395-b4f9-15f6f793f76f service glance] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:19.447458 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e541c25-7523-4eab-b848-c3afc351337f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:19.450815 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e541c25-7523-4eab-b848-c3afc351337f admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:19.450815 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e541c25-7523-4eab-b848-c3afc351337f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:19.450815 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e541c25-7523-4eab-b848-c3afc351337f admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:19.450815 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e541c25-7523-4eab-b848-c3afc351337f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:07:19.450815 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e541c25-7523-4eab-b848-c3afc351337f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:19.453170 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e541c25-7523-4eab-b848-c3afc351337f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:19.470280 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 866/1714] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:07:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:07:19.471450 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e541c25-7523-4eab-b848-c3afc351337f admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:19.474517 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2e541c25-7523-4eab-b848-c3afc351337f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:19.481730 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 849/1715] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:19.482912 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89db3a8-5849-4b84-9188-3d51b01d5e7b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:19.499592 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e541c25-7523-4eab-b848-c3afc351337f admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:19.503714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89db3a8-5849-4b84-9188-3d51b01d5e7b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:19.506620 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e490d8e3-65d2-45b3-90aa-600607598e79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:19.507523 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490d8e3-65d2-45b3-90aa-600607598e79 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:19.508041 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490d8e3-65d2-45b3-90aa-600607598e79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:19.511102 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490d8e3-65d2-45b3-90aa-600607598e79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:19.516633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5f4ac-cf1e-46f5-9d31-a6b60d7c5396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:19.517425 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5f4ac-cf1e-46f5-9d31-a6b60d7c5396 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:19.517717 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5f4ac-cf1e-46f5-9d31-a6b60d7c5396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:19.517988 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5f4ac-cf1e-46f5-9d31-a6b60d7c5396 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:19.519243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5f4ac-cf1e-46f5-9d31-a6b60d7c5396 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:19.525487 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5f4ac-cf1e-46f5-9d31-a6b60d7c5396 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:19.531799 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e490d8e3-65d2-45b3-90aa-600607598e79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:19.535451 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e490d8e3-65d2-45b3-90aa-600607598e79 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:19.548098 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 867/1716] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:19.549681 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e490d8e3-65d2-45b3-90aa-600607598e79 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:19.555965 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e490d8e3-65d2-45b3-90aa-600607598e79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:19.575756 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e490d8e3-65d2-45b3-90aa-600607598e79 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:19.581761 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f083359-7a91-4f92-aabd-120fadce06d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:19.582855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f083359-7a91-4f92-aabd-120fadce06d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:19.583278 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f083359-7a91-4f92-aabd-120fadce06d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:19.584056 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f083359-7a91-4f92-aabd-120fadce06d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:19.602147 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f083359-7a91-4f92-aabd-120fadce06d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:19.602147 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f083359-7a91-4f92-aabd-120fadce06d5 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:19.621250 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 868/1717] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:19.660674 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 850/1718] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:07:19 2022] POST /identity/v3/projects => generated 385 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:19.661831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f083359-7a91-4f92-aabd-120fadce06d5 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:19.679930 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f083359-7a91-4f92-aabd-120fadce06d5 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:19.766964 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821fa3fa-42b6-40a9-9be9-92769e9cd937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:19.767425 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821fa3fa-42b6-40a9-9be9-92769e9cd937 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:19.767839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821fa3fa-42b6-40a9-9be9-92769e9cd937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:19.769040 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821fa3fa-42b6-40a9-9be9-92769e9cd937 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:19.770521 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821fa3fa-42b6-40a9-9be9-92769e9cd937 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:19.773449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821fa3fa-42b6-40a9-9be9-92769e9cd937 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:19.912756 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 869/1719] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:07:19 2022] POST /identity/v3/users => generated 293 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:19.916878 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5f4ac-cf1e-46f5-9d31-a6b60d7c5396 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:19.927572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:19.934816 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5f4ac-cf1e-46f5-9d31-a6b60d7c5396 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:19.952924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832def5c-9749-4dec-b21f-bd263d2fb7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:19.956067 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832def5c-9749-4dec-b21f-bd263d2fb7fd admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:19.956623 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832def5c-9749-4dec-b21f-bd263d2fb7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:19.957241 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832def5c-9749-4dec-b21f-bd263d2fb7fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:19.960049 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832def5c-9749-4dec-b21f-bd263d2fb7fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:19.965383 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832def5c-9749-4dec-b21f-bd263d2fb7fd admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:20.013145 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 851/1720] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:19 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:20.015716 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821fa3fa-42b6-40a9-9be9-92769e9cd937 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:20.035449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821fa3fa-42b6-40a9-9be9-92769e9cd937 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:20.060055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e18123-8fab-43a9-96b1-fb6ac4b76331 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:20.060522 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e18123-8fab-43a9-96b1-fb6ac4b76331 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:20.060904 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e18123-8fab-43a9-96b1-fb6ac4b76331 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:20.062829 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 852/1721] 10.222.0.44 () {54 vars in 912 bytes} [Thu Jun 23 20:07:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:07:20.072196 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8821a16-b9da-432f-a368-4bc5a049d9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:20.076352 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45107a5b-c281-4972-b22a-91ba95d64129 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:20.076352 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45107a5b-c281-4972-b22a-91ba95d64129 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:20.076352 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45107a5b-c281-4972-b22a-91ba95d64129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:20.084775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8821a16-b9da-432f-a368-4bc5a049d9b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:20.085257 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8821a16-b9da-432f-a368-4bc5a049d9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:20.087148 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8821a16-b9da-432f-a368-4bc5a049d9b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:20.088467 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8821a16-b9da-432f-a368-4bc5a049d9b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:20.091967 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8821a16-b9da-432f-a368-4bc5a049d9b7 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:20.132451 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-45107a5b-c281-4972-b22a-91ba95d64129 None None] MFA Rules not processed for user `332fbbefba114da4a01fdd496fff1e31`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:20.137355 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 870/1722] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:20.137733 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-45107a5b-c281-4972-b22a-91ba95d64129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:20.138813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8821a16-b9da-432f-a368-4bc5a049d9b7 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:20.169106 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8821a16-b9da-432f-a368-4bc5a049d9b7 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:20.181007 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a0311-a3e0-46a9-be19-f372d6ba70df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:20.182029 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a0311-a3e0-46a9-be19-f372d6ba70df admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:20.185031 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a0311-a3e0-46a9-be19-f372d6ba70df admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:20.185451 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a0311-a3e0-46a9-be19-f372d6ba70df admin admin] PATH_INFO: `/v3/projects/bc4fa9ef01ed40208899eab624dbfc6a/users/371f624f4ec141df9b439c89450176ea/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:20.224683 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2a0311-a3e0-46a9-be19-f372d6ba70df admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc4fa9ef01ed40208899eab624dbfc6a, user_id=371f624f4ec141df9b439c89450176ea, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:20.224683 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2a0311-a3e0-46a9-be19-f372d6ba70df admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:20.266120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 853/1723] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jun 23 20:07:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:07:20.273684 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45107a5b-c281-4972-b22a-91ba95d64129 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:20.278423 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-45107a5b-c281-4972-b22a-91ba95d64129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:20.296070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45107a5b-c281-4972-b22a-91ba95d64129 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:20.302134 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5676e-eb37-4374-bf19-5936dfd6b5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:20.303184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5676e-eb37-4374-bf19-5936dfd6b5f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:20.304214 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5676e-eb37-4374-bf19-5936dfd6b5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:20.304690 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5676e-eb37-4374-bf19-5936dfd6b5f6 admin admin] PATH_INFO: `/v3/projects/0999954e63ea461f91ff7ddf6fb60086` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:20.306580 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5676e-eb37-4374-bf19-5936dfd6b5f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0999954e63ea461f91ff7ddf6fb60086)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:20.308561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5676e-eb37-4374-bf19-5936dfd6b5f6 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:20.329865 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 871/1724] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:20 2022] PUT /identity/v3/projects/bc4fa9ef01ed40208899eab624dbfc6a/users/371f624f4ec141df9b439c89450176ea/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:20.331630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a0311-a3e0-46a9-be19-f372d6ba70df admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:20.436549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a0311-a3e0-46a9-be19-f372d6ba70df admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:20.444071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbeed02-89a8-4029-ba74-c6869cbe935c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:20.445552 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbeed02-89a8-4029-ba74-c6869cbe935c service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:20.445986 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbeed02-89a8-4029-ba74-c6869cbe935c service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:20.449553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbeed02-89a8-4029-ba74-c6869cbe935c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:20.488179 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-85c5676e-eb37-4374-bf19-5936dfd6b5f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0999954e63ea461f91ff7ddf6fb60086', 'request_id': 'req-85c5676e-eb37-4374-bf19-5936dfd6b5f6'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:20.493774 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbeed02-89a8-4029-ba74-c6869cbe935c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:20.495237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbeed02-89a8-4029-ba74-c6869cbe935c service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:20.514601 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-85c5676e-eb37-4374-bf19-5936dfd6b5f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0999954e63ea461f91ff7ddf6fb60086', 'request_id': 'req-85c5676e-eb37-4374-bf19-5936dfd6b5f6'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:20.538337 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 872/1725] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:20.539665 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4390c8-c8e3-4827-91d7-7afab3409dc2 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:20.539953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4390c8-c8e3-4827-91d7-7afab3409dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:20.540229 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4390c8-c8e3-4827-91d7-7afab3409dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:20.542073 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-85c5676e-eb37-4374-bf19-5936dfd6b5f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0999954e63ea461f91ff7ddf6fb60086', 'request_id': 'req-85c5676e-eb37-4374-bf19-5936dfd6b5f6'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:20.603547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-0d4390c8-c8e3-4827-91d7-7afab3409dc2 None None] MFA Rules not processed for user `371f624f4ec141df9b439c89450176ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:20.608654 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0d4390c8-c8e3-4827-91d7-7afab3409dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:20.671266 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-85c5676e-eb37-4374-bf19-5936dfd6b5f6 admin admin] The token cache is being invalidate because project 0999954e63ea461f91ff7ddf6fb60086 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:07:20.671266 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-85c5676e-eb37-4374-bf19-5936dfd6b5f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:20.671266 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-85c5676e-eb37-4374-bf19-5936dfd6b5f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:20.671266 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 854/1726] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:07:20 2022] DELETE /identity/v3/projects/0999954e63ea461f91ff7ddf6fb60086 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:20.672232 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd82bd-af09-415e-9daa-b57f0f181a16 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:20.673249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd82bd-af09-415e-9daa-b57f0f181a16 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:20.674072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd82bd-af09-415e-9daa-b57f0f181a16 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:20.680783 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 855/1727] 10.222.0.44 () {54 vars in 960 bytes} [Thu Jun 23 20:07:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:07:20.683546 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcad662-2921-4a0d-91ba-970bc202119d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:20.686778 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcad662-2921-4a0d-91ba-970bc202119d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:20.688342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcad662-2921-4a0d-91ba-970bc202119d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:20.745634 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1dcad662-2921-4a0d-91ba-970bc202119d None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:20.751676 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1dcad662-2921-4a0d-91ba-970bc202119d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:20.771576 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 873/1728] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:20.774257 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1907f63-3891-40c9-ac1a-12d90df580a2 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:20.775666 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1907f63-3891-40c9-ac1a-12d90df580a2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:20.776236 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1907f63-3891-40c9-ac1a-12d90df580a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:20.855580 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f1907f63-3891-40c9-ac1a-12d90df580a2 None None] MFA Rules not processed for user `9715d5deeccd419b97cb0bd39e17982f`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:20.858735 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f1907f63-3891-40c9-ac1a-12d90df580a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:20.924348 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 856/1729] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:20.925357 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcad662-2921-4a0d-91ba-970bc202119d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:20.926980 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1dcad662-2921-4a0d-91ba-970bc202119d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:20.947685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcad662-2921-4a0d-91ba-970bc202119d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:20.954580 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1d3176-5526-46ab-a518-79e2e549ce6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:20.956723 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d3176-5526-46ab-a518-79e2e549ce6e admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:20.961459 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d3176-5526-46ab-a518-79e2e549ce6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:20.961459 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1d3176-5526-46ab-a518-79e2e549ce6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:20.961459 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1d3176-5526-46ab-a518-79e2e549ce6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:20.962617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1d3176-5526-46ab-a518-79e2e549ce6e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:21.062377 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 874/1730] 10.222.0.44 () {58 vars in 1057 bytes} [Thu Jun 23 20:07:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:21.063781 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637e0a0-e5d2-4521-bce0-7c26852c7ce1 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:21.064068 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637e0a0-e5d2-4521-bce0-7c26852c7ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:21.064363 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637e0a0-e5d2-4521-bce0-7c26852c7ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:21.093308 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d637e0a0-e5d2-4521-bce0-7c26852c7ce1 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:21.094731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d637e0a0-e5d2-4521-bce0-7c26852c7ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:21.114331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 857/1731] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:07:20 2022] POST /identity/v3/projects => generated 387 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:21.116030 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab7ca2-0758-4451-adbc-db1949829cfe None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:21.116660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab7ca2-0758-4451-adbc-db1949829cfe None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:21.117173 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 875/1732] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:21.117666 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab7ca2-0758-4451-adbc-db1949829cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:21.118346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d637e0a0-e5d2-4521-bce0-7c26852c7ce1 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:21.138140 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d637e0a0-e5d2-4521-bce0-7c26852c7ce1 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:21.140424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d0ab7ca2-0758-4451-adbc-db1949829cfe None None] MFA Rules not processed for user `f041449191c94a5ca5a38c29dd146a15`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:21.142850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d0ab7ca2-0758-4451-adbc-db1949829cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:21.289924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 858/1733] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:21.296985 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab7ca2-0758-4451-adbc-db1949829cfe None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:21.311290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424dfbfc-2e46-4120-bfe7-fc83928c24ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9715d5deeccd419b97cb0bd39e17982f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:21.312552 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424dfbfc-2e46-4120-bfe7-fc83928c24ac service placement] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:21.313146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424dfbfc-2e46-4120-bfe7-fc83928c24ac service placement] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:21.316621 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424dfbfc-2e46-4120-bfe7-fc83928c24ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:21.321100 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-424dfbfc-2e46-4120-bfe7-fc83928c24ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:21.348609 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424dfbfc-2e46-4120-bfe7-fc83928c24ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:21.357738 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424dfbfc-2e46-4120-bfe7-fc83928c24ac service placement] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:21.391195 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 876/1734] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:21.392609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab7ca2-0758-4451-adbc-db1949829cfe None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:21.396757 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424dfbfc-2e46-4120-bfe7-fc83928c24ac service placement] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:21.400202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6f6b3-b46d-4869-acfa-dec048a6073a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:21.400910 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6f6b3-b46d-4869-acfa-dec048a6073a admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:21.401205 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6f6b3-b46d-4869-acfa-dec048a6073a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:21.401456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6f6b3-b46d-4869-acfa-dec048a6073a admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:21.402271 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c6f6b3-b46d-4869-acfa-dec048a6073a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:07:21.402482 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c6f6b3-b46d-4869-acfa-dec048a6073a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:21.405083 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c6f6b3-b46d-4869-acfa-dec048a6073a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:21.419533 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:21.427016 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 859/1735] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:07:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:07:21.428128 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6f6b3-b46d-4869-acfa-dec048a6073a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:21.430940 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-61c6f6b3-b46d-4869-acfa-dec048a6073a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:21.442522 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424dfbfc-2e46-4120-bfe7-fc83928c24ac service placement] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:21.465414 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6f6b3-b46d-4869-acfa-dec048a6073a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:21.470379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79a1da-0056-4dee-9d37-64fe7fa9ae5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:21.471005 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3addf-7b59-4d2f-b069-6cc107975dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:21.471884 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79a1da-0056-4dee-9d37-64fe7fa9ae5e admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:21.472043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3addf-7b59-4d2f-b069-6cc107975dd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:21.472411 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3addf-7b59-4d2f-b069-6cc107975dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:21.472690 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3addf-7b59-4d2f-b069-6cc107975dd3 admin admin] PATH_INFO: `/v3/projects/08c9cba6de694be497659409d33421ba` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:21.473850 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79a1da-0056-4dee-9d37-64fe7fa9ae5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:21.473964 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a3addf-7b59-4d2f-b069-6cc107975dd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08c9cba6de694be497659409d33421ba)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:21.474136 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79a1da-0056-4dee-9d37-64fe7fa9ae5e admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:21.476520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a3addf-7b59-4d2f-b069-6cc107975dd3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:21.477992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d79a1da-0056-4dee-9d37-64fe7fa9ae5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:21.480054 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d79a1da-0056-4dee-9d37-64fe7fa9ae5e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:21.609467 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 877/1736] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:07:21 2022] POST /identity/v3/users => generated 294 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:21.609467 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79a1da-0056-4dee-9d37-64fe7fa9ae5e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:21.650652 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79a1da-0056-4dee-9d37-64fe7fa9ae5e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:21.665623 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b8a3addf-7b59-4d2f-b069-6cc107975dd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08c9cba6de694be497659409d33421ba', 'request_id': 'req-b8a3addf-7b59-4d2f-b069-6cc107975dd3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:21.689495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b8a3addf-7b59-4d2f-b069-6cc107975dd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08c9cba6de694be497659409d33421ba', 'request_id': 'req-b8a3addf-7b59-4d2f-b069-6cc107975dd3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:21.706267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b8a3addf-7b59-4d2f-b069-6cc107975dd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08c9cba6de694be497659409d33421ba', 'request_id': 'req-b8a3addf-7b59-4d2f-b069-6cc107975dd3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:21.718784 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:21.807182 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b8a3addf-7b59-4d2f-b069-6cc107975dd3 admin admin] The token cache is being invalidate because project 08c9cba6de694be497659409d33421ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:07:21.807694 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b8a3addf-7b59-4d2f-b069-6cc107975dd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:21.808692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b8a3addf-7b59-4d2f-b069-6cc107975dd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:21.811292 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 860/1737] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:07:21 2022] DELETE /identity/v3/projects/08c9cba6de694be497659409d33421ba => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:21.812519 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3addf-7b59-4d2f-b069-6cc107975dd3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:21.821523 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b8a3addf-7b59-4d2f-b069-6cc107975dd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:21.841689 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca5011-e9ea-4d35-9068-cb0ce0fc30c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:21.842382 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca5011-e9ea-4d35-9068-cb0ce0fc30c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:21.842664 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca5011-e9ea-4d35-9068-cb0ce0fc30c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:21.842941 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca5011-e9ea-4d35-9068-cb0ce0fc30c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:21.843645 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca5011-e9ea-4d35-9068-cb0ce0fc30c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:21.852662 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca5011-e9ea-4d35-9068-cb0ce0fc30c6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:22.044071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3addf-7b59-4d2f-b069-6cc107975dd3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:22.058931 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c579d-f2fb-44db-bf6a-55abeb218d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:22.061121 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c579d-f2fb-44db-bf6a-55abeb218d52 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:22.061405 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c579d-f2fb-44db-bf6a-55abeb218d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:22.062900 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c579d-f2fb-44db-bf6a-55abeb218d52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:22.063619 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816c579d-f2fb-44db-bf6a-55abeb218d52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:22.066530 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 878/1738] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:07:21 2022] POST /identity/v3/projects => generated 387 bytes in 458 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:22.067290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca5011-e9ea-4d35-9068-cb0ce0fc30c6 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:22.068175 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816c579d-f2fb-44db-bf6a-55abeb218d52 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:22.073606 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a8ca5011-e9ea-4d35-9068-cb0ce0fc30c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:22.092489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 861/1739] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:22.098809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c579d-f2fb-44db-bf6a-55abeb218d52 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:22.105079 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-816c579d-f2fb-44db-bf6a-55abeb218d52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:22.135828 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c579d-f2fb-44db-bf6a-55abeb218d52 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:22.235691 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab76ca-aaba-4e69-b9ee-e6e6e998b00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:22.236968 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab76ca-aaba-4e69-b9ee-e6e6e998b00f admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:22.237845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab76ca-aaba-4e69-b9ee-e6e6e998b00f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:22.238210 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab76ca-aaba-4e69-b9ee-e6e6e998b00f admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:22.239171 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab76ca-aaba-4e69-b9ee-e6e6e998b00f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:22.250617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab76ca-aaba-4e69-b9ee-e6e6e998b00f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:22.264457 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca5011-e9ea-4d35-9068-cb0ce0fc30c6 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:22.291132 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570f2bc-2074-40be-997a-4679da11dec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:22.297549 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:22.298262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570f2bc-2074-40be-997a-4679da11dec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:22.298262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570f2bc-2074-40be-997a-4679da11dec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:22.298262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9570f2bc-2074-40be-997a-4679da11dec7 admin admin] PATH_INFO: `/v3/users/8f4ead08228347f89e7e8b2e427de608` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:22.298262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9570f2bc-2074-40be-997a-4679da11dec7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f4ead08228347f89e7e8b2e427de608)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:22.298570 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9570f2bc-2074-40be-997a-4679da11dec7 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:22.408431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 862/1740] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:07:22 2022] POST /identity/v3/users => generated 294 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:22.409403 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab76ca-aaba-4e69-b9ee-e6e6e998b00f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:22.428869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab76ca-aaba-4e69-b9ee-e6e6e998b00f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:22.439485 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4852611-a400-4c1b-acbf-ebe1264ac698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:22.441353 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4852611-a400-4c1b-acbf-ebe1264ac698 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:22.441831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4852611-a400-4c1b-acbf-ebe1264ac698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:22.442533 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4852611-a400-4c1b-acbf-ebe1264ac698 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:22.443519 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4852611-a400-4c1b-acbf-ebe1264ac698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:22.445365 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4852611-a400-4c1b-acbf-ebe1264ac698 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:22.496474 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 863/1741] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:22.497588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4852611-a400-4c1b-acbf-ebe1264ac698 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:22.531789 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4852611-a400-4c1b-acbf-ebe1264ac698 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:22.549915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039160e5-ef82-45de-ab76-ea8976ebc051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:22.550629 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039160e5-ef82-45de-ab76-ea8976ebc051 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:22.550911 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039160e5-ef82-45de-ab76-ea8976ebc051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:22.551197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039160e5-ef82-45de-ab76-ea8976ebc051 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:22.551914 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039160e5-ef82-45de-ab76-ea8976ebc051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:22.553591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039160e5-ef82-45de-ab76-ea8976ebc051 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:22.598563 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 864/1742] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:22.600787 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039160e5-ef82-45de-ab76-ea8976ebc051 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:22.608578 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9570f2bc-2074-40be-997a-4679da11dec7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f4ead08228347f89e7e8b2e427de608'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:22.612630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9570f2bc-2074-40be-997a-4679da11dec7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f4ead08228347f89e7e8b2e427de608'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:22.626603 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039160e5-ef82-45de-ab76-ea8976ebc051 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:22.632921 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09510c60-8411-4910-94ac-500d5ccafb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:22.633780 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09510c60-8411-4910-94ac-500d5ccafb39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:22.634397 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09510c60-8411-4910-94ac-500d5ccafb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:22.656505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09510c60-8411-4910-94ac-500d5ccafb39 admin admin] PATH_INFO: `/v3/projects/5ae70fd9426f415ea2cca344f8b9d4d8/users/daee89a501184d919325f779fff8c8b4/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:22.702631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09510c60-8411-4910-94ac-500d5ccafb39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ae70fd9426f415ea2cca344f8b9d4d8, user_id=daee89a501184d919325f779fff8c8b4, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:22.711174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09510c60-8411-4910-94ac-500d5ccafb39 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:22.722981 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9570f2bc-2074-40be-997a-4679da11dec7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f4ead08228347f89e7e8b2e427de608'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:22.726592 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9570f2bc-2074-40be-997a-4679da11dec7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f4ead08228347f89e7e8b2e427de608'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:22.754190 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9570f2bc-2074-40be-997a-4679da11dec7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f4ead08228347f89e7e8b2e427de608'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:22.776400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 879/1743] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:07:22 2022] DELETE /identity/v3/users/8f4ead08228347f89e7e8b2e427de608 => generated 0 bytes in 709 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:22.777469 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570f2bc-2074-40be-997a-4679da11dec7 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:22.784015 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9570f2bc-2074-40be-997a-4679da11dec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:22.840133 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 865/1744] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:22 2022] PUT /identity/v3/projects/5ae70fd9426f415ea2cca344f8b9d4d8/users/daee89a501184d919325f779fff8c8b4/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:22.841912 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c41056-9ee1-4ac6-a6ba-12b42927e368 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:22.845328 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c41056-9ee1-4ac6-a6ba-12b42927e368 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:22.845610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c41056-9ee1-4ac6-a6ba-12b42927e368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:22.919527 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9570f2bc-2074-40be-997a-4679da11dec7 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:22.921640 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c7c41056-9ee1-4ac6-a6ba-12b42927e368 None None] MFA Rules not processed for user `8a5b19c8c0d44e1f93bb1dea72884cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:22.926180 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c7c41056-9ee1-4ac6-a6ba-12b42927e368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:23.117471 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec188e2-dd83-409a-8a0f-12e19455f462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:23.118190 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec188e2-dd83-409a-8a0f-12e19455f462 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:23.118472 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec188e2-dd83-409a-8a0f-12e19455f462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:23.118746 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec188e2-dd83-409a-8a0f-12e19455f462 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:23.119865 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec188e2-dd83-409a-8a0f-12e19455f462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:23.121468 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec188e2-dd83-409a-8a0f-12e19455f462 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:23.192081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 866/1745] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:23.194766 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a50366-e98e-4c6e-85cf-f2c0db0956ff None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:23.209583 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 880/1746] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 432 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:23.210652 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec188e2-dd83-409a-8a0f-12e19455f462 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:23.224092 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a50366-e98e-4c6e-85cf-f2c0db0956ff None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:23.224720 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a50366-e98e-4c6e-85cf-f2c0db0956ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:23.257853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec188e2-dd83-409a-8a0f-12e19455f462 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:23.270203 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06cc17-0e92-4a5d-a55d-e5bae875a8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:23.271128 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06cc17-0e92-4a5d-a55d-e5bae875a8ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:23.271387 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06cc17-0e92-4a5d-a55d-e5bae875a8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:23.271829 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06cc17-0e92-4a5d-a55d-e5bae875a8ff admin admin] PATH_INFO: `/v3/projects/78e9971fe45e4dfc9d84a63189a75c63/users/60e50b445fb440a6833d887f29901d90/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:23.307660 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c06cc17-0e92-4a5d-a55d-e5bae875a8ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=78e9971fe45e4dfc9d84a63189a75c63, user_id=60e50b445fb440a6833d887f29901d90, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:23.310988 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c06cc17-0e92-4a5d-a55d-e5bae875a8ff admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:23.380791 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e2a50366-e98e-4c6e-85cf-f2c0db0956ff None None] MFA Rules not processed for user `daee89a501184d919325f779fff8c8b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:23.382691 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e2a50366-e98e-4c6e-85cf-f2c0db0956ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:23.447178 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 881/1747] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:23 2022] PUT /identity/v3/projects/78e9971fe45e4dfc9d84a63189a75c63/users/60e50b445fb440a6833d887f29901d90/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:23.450788 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e3358-852b-4781-8b87-fc5434fbe807 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:23.451181 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e3358-852b-4781-8b87-fc5434fbe807 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:23.453998 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e3358-852b-4781-8b87-fc5434fbe807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:23.545574 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-186e3358-852b-4781-8b87-fc5434fbe807 None None] MFA Rules not processed for user `60e50b445fb440a6833d887f29901d90`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:23.547486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-186e3358-852b-4781-8b87-fc5434fbe807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:23.579661 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 867/1748] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:23.581892 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a50366-e98e-4c6e-85cf-f2c0db0956ff None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:23.583583 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e2a50366-e98e-4c6e-85cf-f2c0db0956ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:23.624614 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a50366-e98e-4c6e-85cf-f2c0db0956ff None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:23.699829 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 882/1749] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:23.705218 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e3358-852b-4781-8b87-fc5434fbe807 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:23.827569 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352afbc6-1e6a-4a41-aee4-06361932bbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:23.829655 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352afbc6-1e6a-4a41-aee4-06361932bbb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:23.834478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352afbc6-1e6a-4a41-aee4-06361932bbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:23.837238 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352afbc6-1e6a-4a41-aee4-06361932bbb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:23.841967 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352afbc6-1e6a-4a41-aee4-06361932bbb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:23.858199 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352afbc6-1e6a-4a41-aee4-06361932bbb1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:23.938337 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e3358-852b-4781-8b87-fc5434fbe807 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:23.948782 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9eaad-0acc-41e6-9cf4-89317e87ec18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:23.949481 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9eaad-0acc-41e6-9cf4-89317e87ec18 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:23.949836 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9eaad-0acc-41e6-9cf4-89317e87ec18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:23.950121 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9eaad-0acc-41e6-9cf4-89317e87ec18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:23.950900 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe9eaad-0acc-41e6-9cf4-89317e87ec18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:23.953157 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe9eaad-0acc-41e6-9cf4-89317e87ec18 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:24.007449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:24.016556 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 868/1750] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:07:23 2022] POST /identity/v3/projects => generated 387 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:07:24.018764 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdf6f6-6aca-45e2-b371-d443ed986442 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:24.019414 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdf6f6-6aca-45e2-b371-d443ed986442 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:24.019703 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bdf6f6-6aca-45e2-b371-d443ed986442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:24.066756 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 883/1751] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:07:23 2022] POST /identity/v3/projects => generated 387 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:24.066756 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9eaad-0acc-41e6-9cf4-89317e87ec18 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:24.067016 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e0bdf6f6-6aca-45e2-b371-d443ed986442 None None] MFA Rules not processed for user `a3be90df779b4ddb8639d825fcb41234`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:24.068028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e0bdf6f6-6aca-45e2-b371-d443ed986442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:24.195044 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 869/1752] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:24.196459 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdf6f6-6aca-45e2-b371-d443ed986442 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:24.198149 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9eaad-0acc-41e6-9cf4-89317e87ec18 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:24.202483 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e0bdf6f6-6aca-45e2-b371-d443ed986442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:24.222008 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562a26e-aa13-410f-b75c-31026b5122a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:24.222799 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562a26e-aa13-410f-b75c-31026b5122a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:24.223395 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562a26e-aa13-410f-b75c-31026b5122a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:24.223560 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562a26e-aa13-410f-b75c-31026b5122a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:24.223974 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bdf6f6-6aca-45e2-b371-d443ed986442 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:24.224670 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3562a26e-aa13-410f-b75c-31026b5122a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:24.245010 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe88dd5-800f-4b5a-bbb4-ef8569955b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:24.245918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe88dd5-800f-4b5a-bbb4-ef8569955b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:24.246201 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe88dd5-800f-4b5a-bbb4-ef8569955b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:24.246403 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3562a26e-aa13-410f-b75c-31026b5122a9 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:24.246505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe88dd5-800f-4b5a-bbb4-ef8569955b9a admin admin] PATH_INFO: `/v3/projects/4cd09a6fd35d41c3997c91c147da928d` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:24.249730 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe88dd5-800f-4b5a-bbb4-ef8569955b9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cd09a6fd35d41c3997c91c147da928d)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:24.255446 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe88dd5-800f-4b5a-bbb4-ef8569955b9a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:24.410861 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 884/1753] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:07:24 2022] POST /identity/v3/users => generated 300 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:24.419247 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562a26e-aa13-410f-b75c-31026b5122a9 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:24.449431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ffe88dd5-800f-4b5a-bbb4-ef8569955b9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cd09a6fd35d41c3997c91c147da928d', 'request_id': 'req-ffe88dd5-800f-4b5a-bbb4-ef8569955b9a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:24.462795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562a26e-aa13-410f-b75c-31026b5122a9 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:24.481585 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ffe88dd5-800f-4b5a-bbb4-ef8569955b9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cd09a6fd35d41c3997c91c147da928d', 'request_id': 'req-ffe88dd5-800f-4b5a-bbb4-ef8569955b9a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:24.507699 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ffe88dd5-800f-4b5a-bbb4-ef8569955b9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cd09a6fd35d41c3997c91c147da928d', 'request_id': 'req-ffe88dd5-800f-4b5a-bbb4-ef8569955b9a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:24.526535 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:24.564995 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6cabfd-e228-423e-a215-37164c59ce3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:24.566205 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6cabfd-e228-423e-a215-37164c59ce3c admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:24.566717 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6cabfd-e228-423e-a215-37164c59ce3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:24.567193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6cabfd-e228-423e-a215-37164c59ce3c admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:24.567958 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6cabfd-e228-423e-a215-37164c59ce3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:24.571440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6cabfd-e228-423e-a215-37164c59ce3c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:24.626912 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ffe88dd5-800f-4b5a-bbb4-ef8569955b9a admin admin] The token cache is being invalidate because project 4cd09a6fd35d41c3997c91c147da928d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:07:24.626912 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ffe88dd5-800f-4b5a-bbb4-ef8569955b9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:24.628563 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ffe88dd5-800f-4b5a-bbb4-ef8569955b9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:24.631405 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 870/1754] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:07:24 2022] DELETE /identity/v3/projects/4cd09a6fd35d41c3997c91c147da928d => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:24.632361 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe88dd5-800f-4b5a-bbb4-ef8569955b9a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:24.637638 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ffe88dd5-800f-4b5a-bbb4-ef8569955b9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:24.679650 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe88dd5-800f-4b5a-bbb4-ef8569955b9a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:24.692503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 885/1755] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:07:24 2022] POST /identity/v3/users => generated 300 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:24.693964 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cc97c-e73a-43d2-974d-364ce74b59a2 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:24.694250 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cc97c-e73a-43d2-974d-364ce74b59a2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:24.694527 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cc97c-e73a-43d2-974d-364ce74b59a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:24.699395 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8999c-39bc-4fd5-a2c2-8a1134b757d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:24.700015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8999c-39bc-4fd5-a2c2-8a1134b757d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:24.700290 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8999c-39bc-4fd5-a2c2-8a1134b757d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:24.700583 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8999c-39bc-4fd5-a2c2-8a1134b757d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:24.701317 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8999c-39bc-4fd5-a2c2-8a1134b757d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:24.703236 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8999c-39bc-4fd5-a2c2-8a1134b757d6 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:24.726197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 871/1756] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:24.730314 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8999c-39bc-4fd5-a2c2-8a1134b757d6 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:24.735167 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-60d8999c-39bc-4fd5-a2c2-8a1134b757d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:24.746885 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a74cc97c-e73a-43d2-974d-364ce74b59a2 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:24.748084 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a74cc97c-e73a-43d2-974d-364ce74b59a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:24.761049 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8999c-39bc-4fd5-a2c2-8a1134b757d6 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:24.767019 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182302c-a090-4ff1-b47f-dda607f381a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:24.770466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182302c-a090-4ff1-b47f-dda607f381a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:24.770466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182302c-a090-4ff1-b47f-dda607f381a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:24.770466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182302c-a090-4ff1-b47f-dda607f381a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:24.770757 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 886/1757] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:24.770843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2182302c-a090-4ff1-b47f-dda607f381a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:24.770930 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74cc97c-e73a-43d2-974d-364ce74b59a2 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:24.771241 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2182302c-a090-4ff1-b47f-dda607f381a9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:24.781506 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74cc97c-e73a-43d2-974d-364ce74b59a2 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:24.788561 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c25fac-8ce9-4aea-8698-bd63e11afbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:24.789469 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c25fac-8ce9-4aea-8698-bd63e11afbad admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:24.789749 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c25fac-8ce9-4aea-8698-bd63e11afbad admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:24.790018 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c25fac-8ce9-4aea-8698-bd63e11afbad admin admin] PATH_INFO: `/v3/projects/ca9ee084981d46afa36ba6c90b84a2f9/users/cd0f2798ace54ccfb94a78080ddc2afe/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:24.793109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 872/1758] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:24.794892 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0016ba5-48a7-4ff0-a845-29a41746d1fb None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:24.795399 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0016ba5-48a7-4ff0-a845-29a41746d1fb None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:24.795931 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0016ba5-48a7-4ff0-a845-29a41746d1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:24.845407 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c25fac-8ce9-4aea-8698-bd63e11afbad admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca9ee084981d46afa36ba6c90b84a2f9, user_id=cd0f2798ace54ccfb94a78080ddc2afe, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:24.849401 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e0016ba5-48a7-4ff0-a845-29a41746d1fb None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:24.850551 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e0016ba5-48a7-4ff0-a845-29a41746d1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:24.852977 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c25fac-8ce9-4aea-8698-bd63e11afbad admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:24.926295 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 873/1759] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:24.932038 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0016ba5-48a7-4ff0-a845-29a41746d1fb None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:24.960882 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0016ba5-48a7-4ff0-a845-29a41746d1fb None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:24.975682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ca4ca-5e26-4b88-817d-cdc06c6bd3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:24.977005 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ca4ca-5e26-4b88-817d-cdc06c6bd3fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:24.979858 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ca4ca-5e26-4b88-817d-cdc06c6bd3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:24.981146 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ca4ca-5e26-4b88-817d-cdc06c6bd3fc admin admin] PATH_INFO: `/v3/projects/fd8c181ae1de41d2a5a997a0d4ee40ae/users/1f68f92a60734306b4f929e5f85e70ae/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:24.988280 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 887/1760] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:24 2022] PUT /identity/v3/projects/ca9ee084981d46afa36ba6c90b84a2f9/users/cd0f2798ace54ccfb94a78080ddc2afe/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:24.989473 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c25fac-8ce9-4aea-8698-bd63e11afbad admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:25.037113 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ca4ca-5e26-4b88-817d-cdc06c6bd3fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd8c181ae1de41d2a5a997a0d4ee40ae, user_id=1f68f92a60734306b4f929e5f85e70ae, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:25.039737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ca4ca-5e26-4b88-817d-cdc06c6bd3fc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:25.161389 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c25fac-8ce9-4aea-8698-bd63e11afbad admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:25.164267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 874/1761] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:24 2022] PUT /identity/v3/projects/fd8c181ae1de41d2a5a997a0d4ee40ae/users/1f68f92a60734306b4f929e5f85e70ae/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:25.165433 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ca4ca-5e26-4b88-817d-cdc06c6bd3fc admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:25.278196 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ca4ca-5e26-4b88-817d-cdc06c6bd3fc admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:25.296628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d0209-eaa5-4a05-82ff-21009a862aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:25.296628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d0209-eaa5-4a05-82ff-21009a862aa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:25.296628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d0209-eaa5-4a05-82ff-21009a862aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:25.296628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d0209-eaa5-4a05-82ff-21009a862aa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:25.297040 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14d0209-eaa5-4a05-82ff-21009a862aa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:25.302821 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14d0209-eaa5-4a05-82ff-21009a862aa5 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:25.372825 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93dd87-bd0a-4153-9f45-93b90854c373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:25.373511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93dd87-bd0a-4153-9f45-93b90854c373 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:25.373790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93dd87-bd0a-4153-9f45-93b90854c373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:25.374058 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93dd87-bd0a-4153-9f45-93b90854c373 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:25.375350 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 875/1762] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:25.375350 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d0209-eaa5-4a05-82ff-21009a862aa5 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:25.378701 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93dd87-bd0a-4153-9f45-93b90854c373 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:07:25.382500 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93dd87-bd0a-4153-9f45-93b90854c373 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:25.385018 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93dd87-bd0a-4153-9f45-93b90854c373 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:25.429774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d0209-eaa5-4a05-82ff-21009a862aa5 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:25.449388 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 888/1763] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:07:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 460 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:07:25.451805 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93dd87-bd0a-4153-9f45-93b90854c373 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:25.451925 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5db30-e5f3-46bf-b79f-717a679c7e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:25.452423 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5db30-e5f3-46bf-b79f-717a679c7e94 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:25.452544 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5db30-e5f3-46bf-b79f-717a679c7e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:25.452979 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5db30-e5f3-46bf-b79f-717a679c7e94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:25.453872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b5db30-e5f3-46bf-b79f-717a679c7e94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:25.456526 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b5db30-e5f3-46bf-b79f-717a679c7e94 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:25.481779 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 876/1764] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:25.482884 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5db30-e5f3-46bf-b79f-717a679c7e94 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:25.485757 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93dd87-bd0a-4153-9f45-93b90854c373 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:25.504872 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53adc7e-8ec4-4b02-819e-c6484b7c5224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:25.506046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53adc7e-8ec4-4b02-819e-c6484b7c5224 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:25.508201 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53adc7e-8ec4-4b02-819e-c6484b7c5224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:25.509051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53adc7e-8ec4-4b02-819e-c6484b7c5224 admin admin] PATH_INFO: `/v3/projects/ca9ee084981d46afa36ba6c90b84a2f9/users/cd0f2798ace54ccfb94a78080ddc2afe/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:25.513819 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5db30-e5f3-46bf-b79f-717a679c7e94 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:25.516326 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53adc7e-8ec4-4b02-819e-c6484b7c5224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca9ee084981d46afa36ba6c90b84a2f9, user_id=cd0f2798ace54ccfb94a78080ddc2afe, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:25.523228 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53adc7e-8ec4-4b02-819e-c6484b7c5224 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:25.551271 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac3bb8-b1b1-4a1f-a65d-2f088a1b2e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:25.552365 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac3bb8-b1b1-4a1f-a65d-2f088a1b2e15 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:25.552804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac3bb8-b1b1-4a1f-a65d-2f088a1b2e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:25.558780 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac3bb8-b1b1-4a1f-a65d-2f088a1b2e15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:25.559354 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac3bb8-b1b1-4a1f-a65d-2f088a1b2e15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:25.564011 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac3bb8-b1b1-4a1f-a65d-2f088a1b2e15 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:25.573605 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 889/1765] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:25 2022] PUT /identity/v3/projects/ca9ee084981d46afa36ba6c90b84a2f9/users/cd0f2798ace54ccfb94a78080ddc2afe/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:25.574968 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53adc7e-8ec4-4b02-819e-c6484b7c5224 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:25.727229 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53adc7e-8ec4-4b02-819e-c6484b7c5224 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:25.736351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f992f0-4691-4109-9092-eeaf104f081c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:25.736351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f992f0-4691-4109-9092-eeaf104f081c admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:25.736351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f992f0-4691-4109-9092-eeaf104f081c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:25.736735 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f992f0-4691-4109-9092-eeaf104f081c admin admin] PATH_INFO: `/v3/projects/fd8c181ae1de41d2a5a997a0d4ee40ae/users/1f68f92a60734306b4f929e5f85e70ae/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:25.739138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f992f0-4691-4109-9092-eeaf104f081c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd8c181ae1de41d2a5a997a0d4ee40ae, user_id=1f68f92a60734306b4f929e5f85e70ae, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:25.742105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f992f0-4691-4109-9092-eeaf104f081c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:25.762597 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 877/1767] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:07:25 2022] POST /identity/v3/projects => generated 401 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:25.764058 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccd234-dbce-4796-be2f-3a12658e6fc3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:25.764376 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccd234-dbce-4796-be2f-3a12658e6fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:25.764752 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccd234-dbce-4796-be2f-3a12658e6fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:25.766573 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 890/1767] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:25 2022] PUT /identity/v3/projects/fd8c181ae1de41d2a5a997a0d4ee40ae/users/1f68f92a60734306b4f929e5f85e70ae/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:25.767523 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f992f0-4691-4109-9092-eeaf104f081c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:25.843497 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-3cccd234-dbce-4796-be2f-3a12658e6fc3 None None] MFA Rules not processed for user `cd0f2798ace54ccfb94a78080ddc2afe`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:25.844985 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3cccd234-dbce-4796-be2f-3a12658e6fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:25.901206 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f992f0-4691-4109-9092-eeaf104f081c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:25.951090 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 878/1768] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:25.951090 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e9924-5e5f-48ea-ba48-29d684df0c36 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:25.951090 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e9924-5e5f-48ea-ba48-29d684df0c36 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:25.951090 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e9924-5e5f-48ea-ba48-29d684df0c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:25.994057 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-014e9924-5e5f-48ea-ba48-29d684df0c36 None None] MFA Rules not processed for user `1f68f92a60734306b4f929e5f85e70ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:25.996482 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-014e9924-5e5f-48ea-ba48-29d684df0c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:26.036635 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0c989-121e-4516-8300-5aa43a78ab89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:26.037770 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0c989-121e-4516-8300-5aa43a78ab89 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:26.039123 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0c989-121e-4516-8300-5aa43a78ab89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:26.039421 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0c989-121e-4516-8300-5aa43a78ab89 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:26.041171 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0c989-121e-4516-8300-5aa43a78ab89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:26.043169 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0c989-121e-4516-8300-5aa43a78ab89 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:26.094525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 879/1769] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:26.095389 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e9924-5e5f-48ea-ba48-29d684df0c36 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:26.097311 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-014e9924-5e5f-48ea-ba48-29d684df0c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:26.166088 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e9924-5e5f-48ea-ba48-29d684df0c36 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:26.182735 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd31a4-abb9-4497-8b6e-1ccee4181b1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:26.183462 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd31a4-abb9-4497-8b6e-1ccee4181b1f service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:26.183744 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd31a4-abb9-4497-8b6e-1ccee4181b1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:26.184011 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd31a4-abb9-4497-8b6e-1ccee4181b1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:26.187997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 891/1770] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:07:25 2022] POST /identity/v3/users => generated 301 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:26.187997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0c989-121e-4516-8300-5aa43a78ab89 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:26.196461 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-16c0c989-121e-4516-8300-5aa43a78ab89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:26.200217 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd31a4-abb9-4497-8b6e-1ccee4181b1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:26.202084 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd31a4-abb9-4497-8b6e-1ccee4181b1f service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:26.209728 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0c989-121e-4516-8300-5aa43a78ab89 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:26.213694 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea850e15-3487-4e91-8675-44e9909e89a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:26.214499 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea850e15-3487-4e91-8675-44e9909e89a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:26.217587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea850e15-3487-4e91-8675-44e9909e89a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:26.217587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea850e15-3487-4e91-8675-44e9909e89a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:26.217817 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 880/1771] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:26 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:26.218008 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd31a4-abb9-4497-8b6e-1ccee4181b1f service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:26.229064 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd31a4-abb9-4497-8b6e-1ccee4181b1f service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:26.231433 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea850e15-3487-4e91-8675-44e9909e89a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:26.232998 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea850e15-3487-4e91-8675-44e9909e89a5 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:26.234533 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782684e-2e89-45bf-b42c-58c8adcbfeee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:26.235714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782684e-2e89-45bf-b42c-58c8adcbfeee admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:26.236234 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782684e-2e89-45bf-b42c-58c8adcbfeee admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:26.237700 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782684e-2e89-45bf-b42c-58c8adcbfeee admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:26.238770 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782684e-2e89-45bf-b42c-58c8adcbfeee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:26.242671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782684e-2e89-45bf-b42c-58c8adcbfeee admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:26.244470 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 892/1772] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:26 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:26.260495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 881/1773] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:26.262914 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea850e15-3487-4e91-8675-44e9909e89a5 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:26.276595 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea850e15-3487-4e91-8675-44e9909e89a5 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:26.279895 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ad0347-ba90-4237-91e6-2113e16b5997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:26.280646 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad0347-ba90-4237-91e6-2113e16b5997 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:26.282193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad0347-ba90-4237-91e6-2113e16b5997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:26.282477 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad0347-ba90-4237-91e6-2113e16b5997 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:26.283547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ad0347-ba90-4237-91e6-2113e16b5997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:26.285219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ad0347-ba90-4237-91e6-2113e16b5997 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:26.311311 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 893/1774] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:26.311999 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782684e-2e89-45bf-b42c-58c8adcbfeee admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:26.342405 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782684e-2e89-45bf-b42c-58c8adcbfeee admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:26.382304 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0bcf3-1391-458c-8588-0bbe6063640a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:26.385087 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0bcf3-1391-458c-8588-0bbe6063640a admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:26.385688 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0bcf3-1391-458c-8588-0bbe6063640a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:26.386133 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0bcf3-1391-458c-8588-0bbe6063640a admin admin] PATH_INFO: `/v3/projects/f1fe03adde8646d5b351251b960d0aab/users/47b1860cf2b24462ac0a23b39626df31/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:26.414362 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af0bcf3-1391-458c-8588-0bbe6063640a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1fe03adde8646d5b351251b960d0aab, user_id=47b1860cf2b24462ac0a23b39626df31, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:26.418069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af0bcf3-1391-458c-8588-0bbe6063640a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:26.490504 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:26.499204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 882/1775] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:26 2022] PUT /identity/v3/projects/f1fe03adde8646d5b351251b960d0aab/users/47b1860cf2b24462ac0a23b39626df31/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:07:26.511051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9480a0-8c52-4074-aaed-4a2f25e6601e None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:26.512442 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9480a0-8c52-4074-aaed-4a2f25e6601e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:26.513213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9480a0-8c52-4074-aaed-4a2f25e6601e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:26.585228 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-bb9480a0-8c52-4074-aaed-4a2f25e6601e None None] MFA Rules not processed for user `47b1860cf2b24462ac0a23b39626df31`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:26.591323 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bb9480a0-8c52-4074-aaed-4a2f25e6601e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:26.838122 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 894/1776] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:26 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:26.841155 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0bcf3-1391-458c-8588-0bbe6063640a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:27.020524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0bcf3-1391-458c-8588-0bbe6063640a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:27.041097 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a2871-7ed7-41a5-b894-2c9139c4b546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:27.041097 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a2871-7ed7-41a5-b894-2c9139c4b546 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:27.041097 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a2871-7ed7-41a5-b894-2c9139c4b546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:27.041097 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a2871-7ed7-41a5-b894-2c9139c4b546 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:27.041422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3a2871-7ed7-41a5-b894-2c9139c4b546 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:27.043663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3a2871-7ed7-41a5-b894-2c9139c4b546 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:27.167009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 883/1777] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:07:26 2022] POST /identity/v3/projects => generated 401 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:27.171081 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9480a0-8c52-4074-aaed-4a2f25e6601e None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:27.191783 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9480a0-8c52-4074-aaed-4a2f25e6601e None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:27.249901 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:27.367676 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d0c73-af7b-46c1-8827-aaad297fff24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:27.368404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d0c73-af7b-46c1-8827-aaad297fff24 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:27.368691 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d0c73-af7b-46c1-8827-aaad297fff24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:27.368967 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d0c73-af7b-46c1-8827-aaad297fff24 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:27.370002 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580d0c73-af7b-46c1-8827-aaad297fff24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:27.373136 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580d0c73-af7b-46c1-8827-aaad297fff24 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:27.498152 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceffcf9-016c-439e-ae4e-cd247c582105 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:27.498695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceffcf9-016c-439e-ae4e-cd247c582105 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:27.499712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceffcf9-016c-439e-ae4e-cd247c582105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:27.512366 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 895/1778] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:07:27 2022] POST /identity/v3/users => generated 307 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:27.517444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d0c73-af7b-46c1-8827-aaad297fff24 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:27.535776 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1ceffcf9-016c-439e-ae4e-cd247c582105 None None] MFA Rules not processed for user `1d37f369749143ce813c693adc5f2ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:27.545406 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1ceffcf9-016c-439e-ae4e-cd247c582105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:27.592697 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d0c73-af7b-46c1-8827-aaad297fff24 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:27.599543 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05165507-11eb-4db0-a93b-0389cb5ad78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:27.601097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05165507-11eb-4db0-a93b-0389cb5ad78d admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:27.601565 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05165507-11eb-4db0-a93b-0389cb5ad78d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:27.602016 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05165507-11eb-4db0-a93b-0389cb5ad78d admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:27.613719 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05165507-11eb-4db0-a93b-0389cb5ad78d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:27.617595 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05165507-11eb-4db0-a93b-0389cb5ad78d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:27.639127 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 884/1779] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jun 23 20:07:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:07:27.645700 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 896/1780] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:27.651481 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceffcf9-016c-439e-ae4e-cd247c582105 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:27.667736 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceffcf9-016c-439e-ae4e-cd247c582105 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:27.672784 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2378b9df-6e7b-4ca2-815a-c463354b1417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:27.676594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378b9df-6e7b-4ca2-815a-c463354b1417 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:27.676884 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378b9df-6e7b-4ca2-815a-c463354b1417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:27.677163 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378b9df-6e7b-4ca2-815a-c463354b1417 admin admin] PATH_INFO: `/v3/projects/e8d5b4557d824645a0ae3b970a6c73e0/users/ace7195bfe634a42bf230804c1d9a74c/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:27.716214 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2378b9df-6e7b-4ca2-815a-c463354b1417 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8d5b4557d824645a0ae3b970a6c73e0, user_id=ace7195bfe634a42bf230804c1d9a74c, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:27.719879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2378b9df-6e7b-4ca2-815a-c463354b1417 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:27.864148 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 885/1781] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:27 2022] PUT /identity/v3/projects/e8d5b4557d824645a0ae3b970a6c73e0/users/ace7195bfe634a42bf230804c1d9a74c/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:27.865597 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05165507-11eb-4db0-a93b-0389cb5ad78d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:27.889869 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05165507-11eb-4db0-a93b-0389cb5ad78d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:28.190785 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21b580-ce6c-4d0e-8aec-48dfe2b2984a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:28.191647 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21b580-ce6c-4d0e-8aec-48dfe2b2984a admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:28.192023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21b580-ce6c-4d0e-8aec-48dfe2b2984a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:28.192422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21b580-ce6c-4d0e-8aec-48dfe2b2984a admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:28.194165 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce21b580-ce6c-4d0e-8aec-48dfe2b2984a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:28.198581 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce21b580-ce6c-4d0e-8aec-48dfe2b2984a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:28.232999 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 897/1782] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:28.237494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2378b9df-6e7b-4ca2-815a-c463354b1417 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:28.261299 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2378b9df-6e7b-4ca2-815a-c463354b1417 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:28.263489 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21b580-ce6c-4d0e-8aec-48dfe2b2984a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:28.265935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ce21b580-ce6c-4d0e-8aec-48dfe2b2984a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:28.297994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4c801-29c1-4463-a994-e28c3fa6168d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:28.299012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4c801-29c1-4463-a994-e28c3fa6168d admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:28.301255 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4c801-29c1-4463-a994-e28c3fa6168d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:28.301647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4c801-29c1-4463-a994-e28c3fa6168d admin admin] PATH_INFO: `/v3/projects/e8d5b4557d824645a0ae3b970a6c73e0/users/ace7195bfe634a42bf230804c1d9a74c/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:28.310608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d4c801-29c1-4463-a994-e28c3fa6168d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8d5b4557d824645a0ae3b970a6c73e0, user_id=ace7195bfe634a42bf230804c1d9a74c, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:28.318542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d4c801-29c1-4463-a994-e28c3fa6168d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:28.350178 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 886/1783] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:28 2022] PUT /identity/v3/projects/e8d5b4557d824645a0ae3b970a6c73e0/users/ace7195bfe634a42bf230804c1d9a74c/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:28.363286 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d58062-7096-41ac-9e93-f6c0181add84 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:28.363616 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d58062-7096-41ac-9e93-f6c0181add84 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:28.363903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d58062-7096-41ac-9e93-f6c0181add84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:28.442379 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-08d58062-7096-41ac-9e93-f6c0181add84 None None] MFA Rules not processed for user `ace7195bfe634a42bf230804c1d9a74c`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:28.444153 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-08d58062-7096-41ac-9e93-f6c0181add84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:28.483823 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21b580-ce6c-4d0e-8aec-48dfe2b2984a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:28.610584 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 887/1784] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:28 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:28.613061 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d58062-7096-41ac-9e93-f6c0181add84 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:28.686522 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388eeeb5-93fe-4e78-8cfe-a95a1f2369de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96db03079816461bb322633f64b9fb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:28.687196 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388eeeb5-93fe-4e78-8cfe-a95a1f2369de service swift] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:28.687481 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388eeeb5-93fe-4e78-8cfe-a95a1f2369de service swift] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:28.687761 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388eeeb5-93fe-4e78-8cfe-a95a1f2369de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:28.732159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d58062-7096-41ac-9e93-f6c0181add84 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:28.736992 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c0620-c87c-46ba-935b-0c64b7ac5a0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:28.737880 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c0620-c87c-46ba-935b-0c64b7ac5a0b service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:28.738293 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c0620-c87c-46ba-935b-0c64b7ac5a0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:28.738705 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c0620-c87c-46ba-935b-0c64b7ac5a0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:28.742331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b83c0620-c87c-46ba-935b-0c64b7ac5a0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:28.799482 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388eeeb5-93fe-4e78-8cfe-a95a1f2369de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:28.801370 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388eeeb5-93fe-4e78-8cfe-a95a1f2369de service swift] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:28.818362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 898/1785] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jun 23 20:07:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 20:07:28.819725 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388eeeb5-93fe-4e78-8cfe-a95a1f2369de service swift] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:28.833758 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388eeeb5-93fe-4e78-8cfe-a95a1f2369de service swift] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:28.839477 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d72479-6d2c-4a16-be6e-5b6d49198efd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:28.840499 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d72479-6d2c-4a16-be6e-5b6d49198efd service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:28.840940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d72479-6d2c-4a16-be6e-5b6d49198efd service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:28.841359 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d72479-6d2c-4a16-be6e-5b6d49198efd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:28.861865 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d72479-6d2c-4a16-be6e-5b6d49198efd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:28.870944 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d72479-6d2c-4a16-be6e-5b6d49198efd service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:28.898760 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 899/1786] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:28 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:28.959622 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c0620-c87c-46ba-935b-0c64b7ac5a0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:28.962736 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c0620-c87c-46ba-935b-0c64b7ac5a0b service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:28.976559 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 888/1787] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:29.713113 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d72479-6d2c-4a16-be6e-5b6d49198efd service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:29.724758 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d72479-6d2c-4a16-be6e-5b6d49198efd service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:29.735400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179dae6e-6bff-4e02-a841-1f37667ad054 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:29.736432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179dae6e-6bff-4e02-a841-1f37667ad054 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:29.736945 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179dae6e-6bff-4e02-a841-1f37667ad054 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:29.739506 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179dae6e-6bff-4e02-a841-1f37667ad054 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:29.923314 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179dae6e-6bff-4e02-a841-1f37667ad054 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:29.928953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179dae6e-6bff-4e02-a841-1f37667ad054 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:29.971999 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 900/1788] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:29 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:29.983777 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1ee8e-c2aa-4871-87c1-d07dfd5f0c3a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:29.983777 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1ee8e-c2aa-4871-87c1-d07dfd5f0c3a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:29.983777 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1ee8e-c2aa-4871-87c1-d07dfd5f0c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:30.023354 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-90a1ee8e-c2aa-4871-87c1-d07dfd5f0c3a None None] MFA Rules not processed for user `332fbbefba114da4a01fdd496fff1e31`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:30.026742 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-90a1ee8e-c2aa-4871-87c1-d07dfd5f0c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:30.158953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 889/1789] 10.222.0.42 () {58 vars in 1015 bytes} [Thu Jun 23 20:07:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:30.190998 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179dae6e-6bff-4e02-a841-1f37667ad054 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:30.192966 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-179dae6e-6bff-4e02-a841-1f37667ad054 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:30.227076 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179dae6e-6bff-4e02-a841-1f37667ad054 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:30.240477 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8888c-ed66-4fb3-9439-5a2c307556aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:30.241406 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8888c-ed66-4fb3-9439-5a2c307556aa service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:30.241833 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8888c-ed66-4fb3-9439-5a2c307556aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:30.242249 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8888c-ed66-4fb3-9439-5a2c307556aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:30.262652 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c8888c-ed66-4fb3-9439-5a2c307556aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:30.264730 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c8888c-ed66-4fb3-9439-5a2c307556aa service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:30.285966 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 901/1790] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:30.361268 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:31.147708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1ee8e-c2aa-4871-87c1-d07dfd5f0c3a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:31.254467 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:31.408494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1ee8e-c2aa-4871-87c1-d07dfd5f0c3a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:31.433599 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56416a97-37bc-46c9-9fed-4ba67822cd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:31.435565 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56416a97-37bc-46c9-9fed-4ba67822cd4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:31.437308 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56416a97-37bc-46c9-9fed-4ba67822cd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:31.438488 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56416a97-37bc-46c9-9fed-4ba67822cd4e admin admin] PATH_INFO: `/v3/users/60e50b445fb440a6833d887f29901d90` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:31.446871 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56416a97-37bc-46c9-9fed-4ba67822cd4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=60e50b445fb440a6833d887f29901d90)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:31.446871 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56416a97-37bc-46c9-9fed-4ba67822cd4e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:31.869077 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-56416a97-37bc-46c9-9fed-4ba67822cd4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60e50b445fb440a6833d887f29901d90'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:31.873963 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-56416a97-37bc-46c9-9fed-4ba67822cd4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60e50b445fb440a6833d887f29901d90'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:31.922007 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-56416a97-37bc-46c9-9fed-4ba67822cd4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60e50b445fb440a6833d887f29901d90'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:31.944408 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-56416a97-37bc-46c9-9fed-4ba67822cd4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60e50b445fb440a6833d887f29901d90'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:32.070234 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-56416a97-37bc-46c9-9fed-4ba67822cd4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60e50b445fb440a6833d887f29901d90'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:32.088454 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 890/1791] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:07:31 2022] DELETE /identity/v3/users/60e50b445fb440a6833d887f29901d90 => generated 0 bytes in 938 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:32.089724 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4bf95-5f54-4071-984b-b08e5c34bb6c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:32.090215 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4bf95-5f54-4071-984b-b08e5c34bb6c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:32.090309 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4bf95-5f54-4071-984b-b08e5c34bb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:32.138182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-89e4bf95-5f54-4071-984b-b08e5c34bb6c None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:32.148629 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-89e4bf95-5f54-4071-984b-b08e5c34bb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:32.361872 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 902/1792] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:32.390629 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56416a97-37bc-46c9-9fed-4ba67822cd4e admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:32.394230 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-56416a97-37bc-46c9-9fed-4ba67822cd4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:32.539685 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:32.589366 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56416a97-37bc-46c9-9fed-4ba67822cd4e admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:32.605904 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7e4d0-b6a4-45f2-b928-299fa2fcf8e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:32.606579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7e4d0-b6a4-45f2-b928-299fa2fcf8e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:32.606914 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7e4d0-b6a4-45f2-b928-299fa2fcf8e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:32.607144 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7e4d0-b6a4-45f2-b928-299fa2fcf8e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:32.636177 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c7e4d0-b6a4-45f2-b928-299fa2fcf8e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:32.637924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c7e4d0-b6a4-45f2-b928-299fa2fcf8e4 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:32.669295 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 891/1793] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:33.219680 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a6b9d-c06f-4ce2-9a5d-117c2973514b None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:33.220628 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a6b9d-c06f-4ce2-9a5d-117c2973514b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:33.222068 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a6b9d-c06f-4ce2-9a5d-117c2973514b None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:33.225803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 903/1794] 10.222.0.44 () {54 vars in 921 bytes} [Thu Jun 23 20:07:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:07:33.238709 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c987da-6ba1-4aee-866e-05bb227c63dd None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:33.239432 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c987da-6ba1-4aee-866e-05bb227c63dd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:33.239955 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c987da-6ba1-4aee-866e-05bb227c63dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:33.276219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a3c987da-6ba1-4aee-866e-05bb227c63dd None None] MFA Rules not processed for user `a63d1d902c024c97a38f7d92f73814d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:33.302905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a3c987da-6ba1-4aee-866e-05bb227c63dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:33.424890 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 892/1795] 10.222.0.44 () {58 vars in 1018 bytes} [Thu Jun 23 20:07:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:33.597086 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a6b9d-c06f-4ce2-9a5d-117c2973514b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:33.599157 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c50a6b9d-c06f-4ce2-9a5d-117c2973514b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:33.624695 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=86,keystone:INSERT=5,keystone:DELETE=9 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:33.638283 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a6b9d-c06f-4ce2-9a5d-117c2973514b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:33.650117 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba97862-373d-417e-8e11-bd574fed3b74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:33.651414 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba97862-373d-417e-8e11-bd574fed3b74 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:33.651926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba97862-373d-417e-8e11-bd574fed3b74 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:33.652377 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba97862-373d-417e-8e11-bd574fed3b74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:33.677345 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba97862-373d-417e-8e11-bd574fed3b74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:33.680677 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba97862-373d-417e-8e11-bd574fed3b74 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:33.699360 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 904/1796] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:34.098672 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:35.028801 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c987da-6ba1-4aee-866e-05bb227c63dd None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:35.032014 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a3c987da-6ba1-4aee-866e-05bb227c63dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:35.055025 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c987da-6ba1-4aee-866e-05bb227c63dd None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:35.060773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf50e7-cead-4002-ac6b-d7852bc27c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:35.067536 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf50e7-cead-4002-ac6b-d7852bc27c25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:35.068137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf50e7-cead-4002-ac6b-d7852bc27c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:35.068595 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf50e7-cead-4002-ac6b-d7852bc27c25 admin admin] PATH_INFO: `/v3/projects/78e9971fe45e4dfc9d84a63189a75c63` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:35.070013 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bf50e7-cead-4002-ac6b-d7852bc27c25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78e9971fe45e4dfc9d84a63189a75c63)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:35.072618 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bf50e7-cead-4002-ac6b-d7852bc27c25 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:35.431499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-17bf50e7-cead-4002-ac6b-d7852bc27c25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78e9971fe45e4dfc9d84a63189a75c63', 'request_id': 'req-17bf50e7-cead-4002-ac6b-d7852bc27c25'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:35.448233 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-17bf50e7-cead-4002-ac6b-d7852bc27c25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78e9971fe45e4dfc9d84a63189a75c63', 'request_id': 'req-17bf50e7-cead-4002-ac6b-d7852bc27c25'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:35.469157 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-17bf50e7-cead-4002-ac6b-d7852bc27c25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78e9971fe45e4dfc9d84a63189a75c63', 'request_id': 'req-17bf50e7-cead-4002-ac6b-d7852bc27c25'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:35.557139 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-17bf50e7-cead-4002-ac6b-d7852bc27c25 admin admin] The token cache is being invalidate because project 78e9971fe45e4dfc9d84a63189a75c63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:07:35.559533 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-17bf50e7-cead-4002-ac6b-d7852bc27c25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:35.559533 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-17bf50e7-cead-4002-ac6b-d7852bc27c25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:35.566489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 893/1797] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:07:35 2022] DELETE /identity/v3/projects/78e9971fe45e4dfc9d84a63189a75c63 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:35.567311 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba97862-373d-417e-8e11-bd574fed3b74 service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:35.593780 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba97862-373d-417e-8e11-bd574fed3b74 service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:35.818381 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42915b8c-3776-42b2-b802-1cc84b6e35c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:35.820504 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42915b8c-3776-42b2-b802-1cc84b6e35c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:35.820504 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42915b8c-3776-42b2-b802-1cc84b6e35c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:35.828579 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42915b8c-3776-42b2-b802-1cc84b6e35c4 admin admin] PATH_INFO: `/v3/users/1f68f92a60734306b4f929e5f85e70ae` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:35.838424 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42915b8c-3776-42b2-b802-1cc84b6e35c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f68f92a60734306b4f929e5f85e70ae)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:35.853883 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42915b8c-3776-42b2-b802-1cc84b6e35c4 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:36.109956 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-42915b8c-3776-42b2-b802-1cc84b6e35c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f68f92a60734306b4f929e5f85e70ae'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:36.110857 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-42915b8c-3776-42b2-b802-1cc84b6e35c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f68f92a60734306b4f929e5f85e70ae'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:36.240860 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-42915b8c-3776-42b2-b802-1cc84b6e35c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f68f92a60734306b4f929e5f85e70ae'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:36.241885 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-42915b8c-3776-42b2-b802-1cc84b6e35c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f68f92a60734306b4f929e5f85e70ae'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:36.262645 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-42915b8c-3776-42b2-b802-1cc84b6e35c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f68f92a60734306b4f929e5f85e70ae'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:36.319338 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 905/1798] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:07:35 2022] DELETE /identity/v3/users/1f68f92a60734306b4f929e5f85e70ae => generated 0 bytes in 752 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:37.529310 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff2843-ec1e-4bd3-a460-c66e765b91c0 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:37.530318 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff2843-ec1e-4bd3-a460-c66e765b91c0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:37.531580 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff2843-ec1e-4bd3-a460-c66e765b91c0 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:37.534745 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 894/1799] 10.222.0.44 () {54 vars in 921 bytes} [Thu Jun 23 20:07:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:07:37.576571 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73982d10-cf38-4db6-a9f8-2972ee926466 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:37.576571 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73982d10-cf38-4db6-a9f8-2972ee926466 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:37.576571 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73982d10-cf38-4db6-a9f8-2972ee926466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:37.609475 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff2843-ec1e-4bd3-a460-c66e765b91c0 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:37.612055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-74ff2843-ec1e-4bd3-a460-c66e765b91c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:37.623351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-73982d10-cf38-4db6-a9f8-2972ee926466 None None] MFA Rules not processed for user `a63d1d902c024c97a38f7d92f73814d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:37.624637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-73982d10-cf38-4db6-a9f8-2972ee926466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:37.810494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff2843-ec1e-4bd3-a460-c66e765b91c0 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:37.819962 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:37.825891 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ce3db-85fd-4ea6-b581-3856fc756ba6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:37.829112 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ce3db-85fd-4ea6-b581-3856fc756ba6 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:37.829112 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ce3db-85fd-4ea6-b581-3856fc756ba6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:37.848289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ce3db-85fd-4ea6-b581-3856fc756ba6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:37.855980 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-671ce3db-85fd-4ea6-b581-3856fc756ba6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:38.015971 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ce3db-85fd-4ea6-b581-3856fc756ba6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:38.018550 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ce3db-85fd-4ea6-b581-3856fc756ba6 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:38.031699 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 906/1800] 10.222.0.44 () {58 vars in 1018 bytes} [Thu Jun 23 20:07:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 458 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:07:38.032883 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73982d10-cf38-4db6-a9f8-2972ee926466 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:38.036766 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-73982d10-cf38-4db6-a9f8-2972ee926466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:38.051331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 895/1801] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 20:07:38.094932 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ce3db-85fd-4ea6-b581-3856fc756ba6 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:38.096786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-671ce3db-85fd-4ea6-b581-3856fc756ba6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:38.181714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73982d10-cf38-4db6-a9f8-2972ee926466 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:38.188010 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70840fb1-aaed-4900-bed1-dc49969ea7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:38.190670 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70840fb1-aaed-4900-bed1-dc49969ea7f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:38.190960 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70840fb1-aaed-4900-bed1-dc49969ea7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:38.191392 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70840fb1-aaed-4900-bed1-dc49969ea7f8 admin admin] PATH_INFO: `/v3/users/47b1860cf2b24462ac0a23b39626df31` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:38.195726 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70840fb1-aaed-4900-bed1-dc49969ea7f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47b1860cf2b24462ac0a23b39626df31)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:38.198071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70840fb1-aaed-4900-bed1-dc49969ea7f8 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:38.230938 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:38.271862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ce3db-85fd-4ea6-b581-3856fc756ba6 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:38.276802 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed9cd5-5215-4f79-8738-5330b4d6716c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:38.277676 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed9cd5-5215-4f79-8738-5330b4d6716c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:38.277956 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed9cd5-5215-4f79-8738-5330b4d6716c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:38.278254 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed9cd5-5215-4f79-8738-5330b4d6716c admin admin] PATH_INFO: `/v3/projects/fd8c181ae1de41d2a5a997a0d4ee40ae` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:38.279514 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed9cd5-5215-4f79-8738-5330b4d6716c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd8c181ae1de41d2a5a997a0d4ee40ae)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:38.281696 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed9cd5-5215-4f79-8738-5330b4d6716c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:38.444975 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-87ed9cd5-5215-4f79-8738-5330b4d6716c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd8c181ae1de41d2a5a997a0d4ee40ae', 'request_id': 'req-87ed9cd5-5215-4f79-8738-5330b4d6716c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:38.453734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-87ed9cd5-5215-4f79-8738-5330b4d6716c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd8c181ae1de41d2a5a997a0d4ee40ae', 'request_id': 'req-87ed9cd5-5215-4f79-8738-5330b4d6716c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:38.460103 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-70840fb1-aaed-4900-bed1-dc49969ea7f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47b1860cf2b24462ac0a23b39626df31'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:38.461236 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-70840fb1-aaed-4900-bed1-dc49969ea7f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47b1860cf2b24462ac0a23b39626df31'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:38.475985 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-87ed9cd5-5215-4f79-8738-5330b4d6716c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd8c181ae1de41d2a5a997a0d4ee40ae', 'request_id': 'req-87ed9cd5-5215-4f79-8738-5330b4d6716c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:38.588743 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-70840fb1-aaed-4900-bed1-dc49969ea7f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47b1860cf2b24462ac0a23b39626df31'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:38.590863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-70840fb1-aaed-4900-bed1-dc49969ea7f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47b1860cf2b24462ac0a23b39626df31'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:38.617158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-70840fb1-aaed-4900-bed1-dc49969ea7f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47b1860cf2b24462ac0a23b39626df31'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:38.661671 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 907/1802] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:07:38 2022] DELETE /identity/v3/users/47b1860cf2b24462ac0a23b39626df31 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:38.664045 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70840fb1-aaed-4900-bed1-dc49969ea7f8 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:38.669342 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-70840fb1-aaed-4900-bed1-dc49969ea7f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:38.804104 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70840fb1-aaed-4900-bed1-dc49969ea7f8 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:38.812411 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08010fe5-c80b-4522-ab95-4e9ddbae3d71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:38.838406 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08010fe5-c80b-4522-ab95-4e9ddbae3d71 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:38.838406 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08010fe5-c80b-4522-ab95-4e9ddbae3d71 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:38.838406 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08010fe5-c80b-4522-ab95-4e9ddbae3d71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:38.838406 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-08010fe5-c80b-4522-ab95-4e9ddbae3d71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:38.838406 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08010fe5-c80b-4522-ab95-4e9ddbae3d71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:38.838406 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08010fe5-c80b-4522-ab95-4e9ddbae3d71 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:38.853672 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 908/1803] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:38.855284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb1b6e-9360-497a-a1e0-c9ad0bee2328 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:38.855533 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb1b6e-9360-497a-a1e0-c9ad0bee2328 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:38.855856 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb1b6e-9360-497a-a1e0-c9ad0bee2328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:38.875255 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3bcb1b6e-9360-497a-a1e0-c9ad0bee2328 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:38.876519 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3bcb1b6e-9360-497a-a1e0-c9ad0bee2328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:38.956207 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 909/1804] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:38.967993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb1b6e-9360-497a-a1e0-c9ad0bee2328 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:38.969625 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3bcb1b6e-9360-497a-a1e0-c9ad0bee2328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:39.028664 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb1b6e-9360-497a-a1e0-c9ad0bee2328 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:39.032438 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3d87d-1ac2-4459-baa8-c97a6b38489c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:39.033008 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3d87d-1ac2-4459-baa8-c97a6b38489c service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:39.033934 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3d87d-1ac2-4459-baa8-c97a6b38489c service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:39.034216 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3d87d-1ac2-4459-baa8-c97a6b38489c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:39.044213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f3d87d-1ac2-4459-baa8-c97a6b38489c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:39.045716 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f3d87d-1ac2-4459-baa8-c97a6b38489c service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:39.055687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 910/1805] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:39.092611 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-87ed9cd5-5215-4f79-8738-5330b4d6716c admin admin] The token cache is being invalidate because project fd8c181ae1de41d2a5a997a0d4ee40ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:07:39.093177 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-87ed9cd5-5215-4f79-8738-5330b4d6716c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:39.093637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-87ed9cd5-5215-4f79-8738-5330b4d6716c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:39.095242 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 896/1806] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:07:38 2022] DELETE /identity/v3/projects/fd8c181ae1de41d2a5a997a0d4ee40ae => generated 0 bytes in 1001 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:07:39.101680 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8fc8c-aaab-4385-ac81-e5932718da01 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:39.101966 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8fc8c-aaab-4385-ac81-e5932718da01 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:39.102241 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8fc8c-aaab-4385-ac81-e5932718da01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:39.120073 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-47c8fc8c-aaab-4385-ac81-e5932718da01 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:39.121302 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-47c8fc8c-aaab-4385-ac81-e5932718da01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:39.136335 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 911/1807] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:39.140149 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c5488-5f4b-49e4-9dcb-e5955f4efbcf None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:39.140464 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c5488-5f4b-49e4-9dcb-e5955f4efbcf None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:39.140745 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c5488-5f4b-49e4-9dcb-e5955f4efbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:39.159847 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8d7c5488-5f4b-49e4-9dcb-e5955f4efbcf None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:39.161086 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8d7c5488-5f4b-49e4-9dcb-e5955f4efbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:39.169008 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 897/1808] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:39.173190 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8fc8c-aaab-4385-ac81-e5932718da01 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:39.182838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8fc8c-aaab-4385-ac81-e5932718da01 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:39.187296 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab58e94-9c5c-4ce1-835d-7e754ea3e3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:39.187935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab58e94-9c5c-4ce1-835d-7e754ea3e3c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:39.188214 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab58e94-9c5c-4ce1-835d-7e754ea3e3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:39.188518 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab58e94-9c5c-4ce1-835d-7e754ea3e3c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:39.189235 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab58e94-9c5c-4ce1-835d-7e754ea3e3c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:07:39.189654 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab58e94-9c5c-4ce1-835d-7e754ea3e3c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:39.192616 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab58e94-9c5c-4ce1-835d-7e754ea3e3c6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:39.208619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 912/1809] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:07:39.213058 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c5488-5f4b-49e4-9dcb-e5955f4efbcf None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:39.222820 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c5488-5f4b-49e4-9dcb-e5955f4efbcf None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:39.227543 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebb3ad-95a6-4da5-aa71-d1e18ce9ffdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:39.228197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebb3ad-95a6-4da5-aa71-d1e18ce9ffdc admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:39.228537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebb3ad-95a6-4da5-aa71-d1e18ce9ffdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:39.228830 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ebb3ad-95a6-4da5-aa71-d1e18ce9ffdc admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:39.229518 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ebb3ad-95a6-4da5-aa71-d1e18ce9ffdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:39.231445 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ebb3ad-95a6-4da5-aa71-d1e18ce9ffdc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:39.407120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 898/1810] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:07:39 2022] POST /identity/v3/projects => generated 373 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:39.410191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab58e94-9c5c-4ce1-835d-7e754ea3e3c6 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:39.424955 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab58e94-9c5c-4ce1-835d-7e754ea3e3c6 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:39.501655 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5832b7-e7e7-4efe-ab4f-cd829ce1ce48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:39.502961 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5832b7-e7e7-4efe-ab4f-cd829ce1ce48 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:39.503197 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5832b7-e7e7-4efe-ab4f-cd829ce1ce48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:39.504097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5832b7-e7e7-4efe-ab4f-cd829ce1ce48 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:39.504839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5832b7-e7e7-4efe-ab4f-cd829ce1ce48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:39.506832 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5832b7-e7e7-4efe-ab4f-cd829ce1ce48 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:39.636274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 913/1811] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:07:39 2022] POST /identity/v3/users => generated 287 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:39.640431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebb3ad-95a6-4da5-aa71-d1e18ce9ffdc admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:39.651058 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ebb3ad-95a6-4da5-aa71-d1e18ce9ffdc admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:39.655825 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e031691a-7734-4d4c-b081-52927f58c023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:39.656523 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031691a-7734-4d4c-b081-52927f58c023 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:39.656920 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031691a-7734-4d4c-b081-52927f58c023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:39.657201 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e031691a-7734-4d4c-b081-52927f58c023 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:39.658225 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e031691a-7734-4d4c-b081-52927f58c023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:39.659754 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e031691a-7734-4d4c-b081-52927f58c023 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:39.671286 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 899/1812] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:39.676470 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5832b7-e7e7-4efe-ab4f-cd829ce1ce48 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:39.686998 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5832b7-e7e7-4efe-ab4f-cd829ce1ce48 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:39.703692 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098ca65-8dba-4f00-b136-7563fb79a529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:39.703692 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098ca65-8dba-4f00-b136-7563fb79a529 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:39.703692 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098ca65-8dba-4f00-b136-7563fb79a529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:39.703692 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5098ca65-8dba-4f00-b136-7563fb79a529 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:39.704052 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5098ca65-8dba-4f00-b136-7563fb79a529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:39.704052 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5098ca65-8dba-4f00-b136-7563fb79a529 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:39.711998 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 914/1813] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:39.715061 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e031691a-7734-4d4c-b081-52927f58c023 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:39.725437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e031691a-7734-4d4c-b081-52927f58c023 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:39.729967 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06fbc0-8786-4e07-bfc4-065fadf34087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:39.730868 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06fbc0-8786-4e07-bfc4-065fadf34087 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:39.731124 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06fbc0-8786-4e07-bfc4-065fadf34087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:39.731398 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06fbc0-8786-4e07-bfc4-065fadf34087 admin admin] PATH_INFO: `/v3/projects/9ac154e6a7ac45adb7bcb0736b0f1b77/users/381a7d48b683470785bef900084b6b53/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:39.746877 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06fbc0-8786-4e07-bfc4-065fadf34087 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ac154e6a7ac45adb7bcb0736b0f1b77, user_id=381a7d48b683470785bef900084b6b53, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:39.749593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06fbc0-8786-4e07-bfc4-065fadf34087 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:39.858227 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098ca65-8dba-4f00-b136-7563fb79a529 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:39.860262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5098ca65-8dba-4f00-b136-7563fb79a529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:39.874874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5098ca65-8dba-4f00-b136-7563fb79a529 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:39.881005 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c81d6e-6609-42e5-8bb8-5cf955e9f294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:39.883269 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c81d6e-6609-42e5-8bb8-5cf955e9f294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:39.883269 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c81d6e-6609-42e5-8bb8-5cf955e9f294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:39.883269 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c81d6e-6609-42e5-8bb8-5cf955e9f294 admin admin] PATH_INFO: `/v3/projects/f1fe03adde8646d5b351251b960d0aab` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:39.894877 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c81d6e-6609-42e5-8bb8-5cf955e9f294 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1fe03adde8646d5b351251b960d0aab)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:39.896905 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c81d6e-6609-42e5-8bb8-5cf955e9f294 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:39.949136 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 900/1814] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:39 2022] PUT /identity/v3/projects/9ac154e6a7ac45adb7bcb0736b0f1b77/users/381a7d48b683470785bef900084b6b53/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:39.968214 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5c569-d7c7-4185-87f9-d445d2270ec0 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:39.968666 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5c569-d7c7-4185-87f9-d445d2270ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:39.969012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5c569-d7c7-4185-87f9-d445d2270ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:39.996480 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1be5c569-d7c7-4185-87f9-d445d2270ec0 None None] MFA Rules not processed for user `381a7d48b683470785bef900084b6b53`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:39.997809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1be5c569-d7c7-4185-87f9-d445d2270ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:40.064931 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 901/1815] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:07:40.076355 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be5c569-d7c7-4185-87f9-d445d2270ec0 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:40.077688 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1be5c569-d7c7-4185-87f9-d445d2270ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:40.115447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-16c81d6e-6609-42e5-8bb8-5cf955e9f294 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1fe03adde8646d5b351251b960d0aab', 'request_id': 'req-16c81d6e-6609-42e5-8bb8-5cf955e9f294'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:40.134068 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be5c569-d7c7-4185-87f9-d445d2270ec0 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:40.181470 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32496f-3654-42c3-9b3c-e423e58502bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:40.182112 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32496f-3654-42c3-9b3c-e423e58502bb service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:40.182391 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32496f-3654-42c3-9b3c-e423e58502bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:40.182662 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32496f-3654-42c3-9b3c-e423e58502bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:40.241539 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce32496f-3654-42c3-9b3c-e423e58502bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:40.242984 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce32496f-3654-42c3-9b3c-e423e58502bb service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:40.251828 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 902/1816] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:40.270563 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32496f-3654-42c3-9b3c-e423e58502bb service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:40.286159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-16c81d6e-6609-42e5-8bb8-5cf955e9f294 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1fe03adde8646d5b351251b960d0aab', 'request_id': 'req-16c81d6e-6609-42e5-8bb8-5cf955e9f294'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:40.292062 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-16c81d6e-6609-42e5-8bb8-5cf955e9f294 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1fe03adde8646d5b351251b960d0aab', 'request_id': 'req-16c81d6e-6609-42e5-8bb8-5cf955e9f294'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:40.308371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-16c81d6e-6609-42e5-8bb8-5cf955e9f294 admin admin] The token cache is being invalidate because project f1fe03adde8646d5b351251b960d0aab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:07:40.308371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-16c81d6e-6609-42e5-8bb8-5cf955e9f294 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:40.308371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-16c81d6e-6609-42e5-8bb8-5cf955e9f294 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:40.308371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 915/1817] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:07:39 2022] DELETE /identity/v3/projects/f1fe03adde8646d5b351251b960d0aab => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:40.310545 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c81d6e-6609-42e5-8bb8-5cf955e9f294 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:40.311839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-16c81d6e-6609-42e5-8bb8-5cf955e9f294 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:40.335141 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32496f-3654-42c3-9b3c-e423e58502bb service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:40.340491 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a549a-ae46-4eff-b08a-3645373b5951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:40.341354 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a549a-ae46-4eff-b08a-3645373b5951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:40.342389 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a549a-ae46-4eff-b08a-3645373b5951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:40.342389 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a549a-ae46-4eff-b08a-3645373b5951 admin admin] PATH_INFO: `/v3/users/381a7d48b683470785bef900084b6b53` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:40.343215 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a549a-ae46-4eff-b08a-3645373b5951 admin admin] RBAC: Authorizing `identity:delete_user(user_id=381a7d48b683470785bef900084b6b53)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:40.345242 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a549a-ae46-4eff-b08a-3645373b5951 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:40.384057 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c81d6e-6609-42e5-8bb8-5cf955e9f294 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:40.399725 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009bbee-bb33-4970-b9db-61a3c48684e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:40.399725 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009bbee-bb33-4970-b9db-61a3c48684e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:40.399725 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009bbee-bb33-4970-b9db-61a3c48684e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:40.399725 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009bbee-bb33-4970-b9db-61a3c48684e6 admin admin] PATH_INFO: `/v3/users/ace7195bfe634a42bf230804c1d9a74c` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:40.400092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1009bbee-bb33-4970-b9db-61a3c48684e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ace7195bfe634a42bf230804c1d9a74c)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:40.400092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1009bbee-bb33-4970-b9db-61a3c48684e6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:40.608642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-093a549a-ae46-4eff-b08a-3645373b5951 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '381a7d48b683470785bef900084b6b53'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:40.609607 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-093a549a-ae46-4eff-b08a-3645373b5951 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '381a7d48b683470785bef900084b6b53'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:40.622744 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-093a549a-ae46-4eff-b08a-3645373b5951 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '381a7d48b683470785bef900084b6b53'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:40.630677 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-093a549a-ae46-4eff-b08a-3645373b5951 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '381a7d48b683470785bef900084b6b53'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:40.666865 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1009bbee-bb33-4970-b9db-61a3c48684e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ace7195bfe634a42bf230804c1d9a74c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:40.667960 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1009bbee-bb33-4970-b9db-61a3c48684e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ace7195bfe634a42bf230804c1d9a74c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:40.732685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-093a549a-ae46-4eff-b08a-3645373b5951 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '381a7d48b683470785bef900084b6b53'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:40.741970 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 903/1818] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:07:40 2022] DELETE /identity/v3/users/381a7d48b683470785bef900084b6b53 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:07:40.758013 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58df21-82be-40ba-82c3-5f8308df8712 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:40.758280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58df21-82be-40ba-82c3-5f8308df8712 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:40.758543 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58df21-82be-40ba-82c3-5f8308df8712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:40.778932 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-bb58df21-82be-40ba-82c3-5f8308df8712 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:40.780902 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-bb58df21-82be-40ba-82c3-5f8308df8712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:40.840977 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1009bbee-bb33-4970-b9db-61a3c48684e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ace7195bfe634a42bf230804c1d9a74c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:40.841887 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1009bbee-bb33-4970-b9db-61a3c48684e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ace7195bfe634a42bf230804c1d9a74c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:40.866387 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1009bbee-bb33-4970-b9db-61a3c48684e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ace7195bfe634a42bf230804c1d9a74c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:40.883952 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 916/1819] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:07:40 2022] DELETE /identity/v3/users/ace7195bfe634a42bf230804c1d9a74c => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:07:40.906228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 904/1820] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:40.932096 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009bbee-bb33-4970-b9db-61a3c48684e6 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:40.933725 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1009bbee-bb33-4970-b9db-61a3c48684e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:41.046510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009bbee-bb33-4970-b9db-61a3c48684e6 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:41.052565 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8f7b4-0cdb-4ccf-9674-69a9e6d19743 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:41.053535 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8f7b4-0cdb-4ccf-9674-69a9e6d19743 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:41.054204 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8f7b4-0cdb-4ccf-9674-69a9e6d19743 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:41.054204 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8f7b4-0cdb-4ccf-9674-69a9e6d19743 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:41.130053 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a8f7b4-0cdb-4ccf-9674-69a9e6d19743 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:41.131481 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a8f7b4-0cdb-4ccf-9674-69a9e6d19743 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:41.143370 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 917/1821] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:41.745242 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58df21-82be-40ba-82c3-5f8308df8712 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:41.746781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-bb58df21-82be-40ba-82c3-5f8308df8712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:41.771398 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58df21-82be-40ba-82c3-5f8308df8712 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:41.779931 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f1c61-8782-499d-b20c-4197b42e2182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:41.781015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f1c61-8782-499d-b20c-4197b42e2182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:41.781417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f1c61-8782-499d-b20c-4197b42e2182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:41.781717 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f1c61-8782-499d-b20c-4197b42e2182 admin admin] PATH_INFO: `/v3/projects/e8d5b4557d824645a0ae3b970a6c73e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:41.783112 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f1c61-8782-499d-b20c-4197b42e2182 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8d5b4557d824645a0ae3b970a6c73e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:41.785236 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f1c61-8782-499d-b20c-4197b42e2182 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:41.865862 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8f7b4-0cdb-4ccf-9674-69a9e6d19743 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:41.867383 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c1a8f7b4-0cdb-4ccf-9674-69a9e6d19743 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:41.880518 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8f7b4-0cdb-4ccf-9674-69a9e6d19743 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:41.885443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c9531-b0f3-4355-a633-5ac7c79fc25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:41.889706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c9531-b0f3-4355-a633-5ac7c79fc25d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:41.889706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c9531-b0f3-4355-a633-5ac7c79fc25d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:41.889706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c9531-b0f3-4355-a633-5ac7c79fc25d admin admin] PATH_INFO: `/v3/projects/9ac154e6a7ac45adb7bcb0736b0f1b77` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:41.889706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c9531-b0f3-4355-a633-5ac7c79fc25d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ac154e6a7ac45adb7bcb0736b0f1b77)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:41.890505 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c9531-b0f3-4355-a633-5ac7c79fc25d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:41.907116 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3b0f1c61-8782-499d-b20c-4197b42e2182 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8d5b4557d824645a0ae3b970a6c73e0', 'request_id': 'req-3b0f1c61-8782-499d-b20c-4197b42e2182'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:41.913473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3b0f1c61-8782-499d-b20c-4197b42e2182 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8d5b4557d824645a0ae3b970a6c73e0', 'request_id': 'req-3b0f1c61-8782-499d-b20c-4197b42e2182'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:41.918961 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3b0f1c61-8782-499d-b20c-4197b42e2182 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8d5b4557d824645a0ae3b970a6c73e0', 'request_id': 'req-3b0f1c61-8782-499d-b20c-4197b42e2182'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:42.047239 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3b0f1c61-8782-499d-b20c-4197b42e2182 admin admin] The token cache is being invalidate because project e8d5b4557d824645a0ae3b970a6c73e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:07:42.047537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3b0f1c61-8782-499d-b20c-4197b42e2182 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:42.048182 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3b0f1c61-8782-499d-b20c-4197b42e2182 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:42.050622 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 905/1822] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:07:41 2022] DELETE /identity/v3/projects/e8d5b4557d824645a0ae3b970a6c73e0 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:42.058169 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20b344-d2f0-4645-b013-00545875b440 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:42.058454 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20b344-d2f0-4645-b013-00545875b440 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:42.058726 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20b344-d2f0-4645-b013-00545875b440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:42.062887 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c80c9531-b0f3-4355-a633-5ac7c79fc25d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ac154e6a7ac45adb7bcb0736b0f1b77', 'request_id': 'req-c80c9531-b0f3-4355-a633-5ac7c79fc25d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:42.079553 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5d20b344-d2f0-4645-b013-00545875b440 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:42.080805 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5d20b344-d2f0-4645-b013-00545875b440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:42.138036 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 906/1823] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:42.145699 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75ef18-f211-4284-90b6-de7dd107946f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:42.146040 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75ef18-f211-4284-90b6-de7dd107946f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:42.146266 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75ef18-f211-4284-90b6-de7dd107946f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:42.149575 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c80c9531-b0f3-4355-a633-5ac7c79fc25d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ac154e6a7ac45adb7bcb0736b0f1b77', 'request_id': 'req-c80c9531-b0f3-4355-a633-5ac7c79fc25d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:42.156289 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c80c9531-b0f3-4355-a633-5ac7c79fc25d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ac154e6a7ac45adb7bcb0736b0f1b77', 'request_id': 'req-c80c9531-b0f3-4355-a633-5ac7c79fc25d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:42.167395 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-2a75ef18-f211-4284-90b6-de7dd107946f None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:42.167553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c80c9531-b0f3-4355-a633-5ac7c79fc25d admin admin] The token cache is being invalidate because project 9ac154e6a7ac45adb7bcb0736b0f1b77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:07:42.167836 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c80c9531-b0f3-4355-a633-5ac7c79fc25d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:42.168501 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c80c9531-b0f3-4355-a633-5ac7c79fc25d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:42.168624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2a75ef18-f211-4284-90b6-de7dd107946f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:42.170117 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 918/1824] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:07:41 2022] DELETE /identity/v3/projects/9ac154e6a7ac45adb7bcb0736b0f1b77 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:07:42.177084 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76e79a-52c1-4fdc-947c-59207da39723 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:42.177384 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76e79a-52c1-4fdc-947c-59207da39723 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:42.177667 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76e79a-52c1-4fdc-947c-59207da39723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:42.179121 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 907/1825] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:07:42.183072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75ef18-f211-4284-90b6-de7dd107946f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:42.192447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75ef18-f211-4284-90b6-de7dd107946f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:42.196114 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-fe76e79a-52c1-4fdc-947c-59207da39723 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:42.197307 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7044644-9381-4210-9c0d-f9d6ed48341f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:42.197619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fe76e79a-52c1-4fdc-947c-59207da39723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:42.197941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7044644-9381-4210-9c0d-f9d6ed48341f admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:42.198224 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7044644-9381-4210-9c0d-f9d6ed48341f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:42.198498 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7044644-9381-4210-9c0d-f9d6ed48341f admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:42.199212 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7044644-9381-4210-9c0d-f9d6ed48341f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:07:42.199530 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7044644-9381-4210-9c0d-f9d6ed48341f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:42.201131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7044644-9381-4210-9c0d-f9d6ed48341f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:42.212118 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 919/1826] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:07:42.213623 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c82db-cc2c-46bc-8c0a-acb4c91864a7 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:42.213919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c82db-cc2c-46bc-8c0a-acb4c91864a7 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:42.214204 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7c82db-cc2c-46bc-8c0a-acb4c91864a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:42.215959 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 908/1827] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:07:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 20:07:42.231690 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7044644-9381-4210-9c0d-f9d6ed48341f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:42.233119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-cf7c82db-cc2c-46bc-8c0a-acb4c91864a7 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:42.234190 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-cf7c82db-cc2c-46bc-8c0a-acb4c91864a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:42.246477 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7044644-9381-4210-9c0d-f9d6ed48341f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:42.247994 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 920/1828] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:42.254525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e5c51-9858-4535-b6f3-3431c1b165ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:42.254525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e5c51-9858-4535-b6f3-3431c1b165ef admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:42.254525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e5c51-9858-4535-b6f3-3431c1b165ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:42.254525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e5c51-9858-4535-b6f3-3431c1b165ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:42.255209 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e5c51-9858-4535-b6f3-3431c1b165ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:42.255705 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c82db-cc2c-46bc-8c0a-acb4c91864a7 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:42.257357 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e5c51-9858-4535-b6f3-3431c1b165ef admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:42.266012 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7c82db-cc2c-46bc-8c0a-acb4c91864a7 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:42.276962 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a528a-0887-41d5-b509-d85349d13f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:42.277773 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a528a-0887-41d5-b509-d85349d13f6b admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:42.278052 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a528a-0887-41d5-b509-d85349d13f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:42.278333 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9a528a-0887-41d5-b509-d85349d13f6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:42.279129 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a528a-0887-41d5-b509-d85349d13f6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:07:42.279432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a528a-0887-41d5-b509-d85349d13f6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:42.281184 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9a528a-0887-41d5-b509-d85349d13f6b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:42.281585 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:42.294389 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 921/1829] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:07:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 20:07:42.298431 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a528a-0887-41d5-b509-d85349d13f6b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:42.309159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9a528a-0887-41d5-b509-d85349d13f6b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:42.314808 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710530c-b438-4f28-b50b-7d8b584aa94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:42.315605 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710530c-b438-4f28-b50b-7d8b584aa94d admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:42.315984 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710530c-b438-4f28-b50b-7d8b584aa94d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:42.316389 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710530c-b438-4f28-b50b-7d8b584aa94d admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:42.317533 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3710530c-b438-4f28-b50b-7d8b584aa94d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:42.319599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3710530c-b438-4f28-b50b-7d8b584aa94d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:42.336695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 909/1830] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:07:42 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:07:42.340479 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e5c51-9858-4535-b6f3-3431c1b165ef admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:42.407524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e5c51-9858-4535-b6f3-3431c1b165ef admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:42.412836 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1bf5dd-c0f4-4643-b114-177c54f5cafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:42.413567 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1bf5dd-c0f4-4643-b114-177c54f5cafc admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:42.413860 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1bf5dd-c0f4-4643-b114-177c54f5cafc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:42.414136 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1bf5dd-c0f4-4643-b114-177c54f5cafc admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:42.414936 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1bf5dd-c0f4-4643-b114-177c54f5cafc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:42.417081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1bf5dd-c0f4-4643-b114-177c54f5cafc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:42.442997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 922/1831] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:07:42 2022] POST /identity/v3/projects => generated 373 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:07:42.454434 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710530c-b438-4f28-b50b-7d8b584aa94d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:42.506900 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 910/1832] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:07:42 2022] POST /identity/v3/users => generated 292 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:07:42.507977 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1bf5dd-c0f4-4643-b114-177c54f5cafc admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:42.517186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710530c-b438-4f28-b50b-7d8b584aa94d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:42.517740 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1bf5dd-c0f4-4643-b114-177c54f5cafc admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:42.522199 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023cd94-3510-46bd-8aa4-0acc10b99634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:42.522536 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb05929-5b69-4d65-b48b-c5d9be1a77ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:42.523023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023cd94-3510-46bd-8aa4-0acc10b99634 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:42.523204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb05929-5b69-4d65-b48b-c5d9be1a77ff admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:42.523485 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb05929-5b69-4d65-b48b-c5d9be1a77ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:42.523615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023cd94-3510-46bd-8aa4-0acc10b99634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:42.523758 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb05929-5b69-4d65-b48b-c5d9be1a77ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:42.524026 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023cd94-3510-46bd-8aa4-0acc10b99634 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:42.524511 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb05929-5b69-4d65-b48b-c5d9be1a77ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:42.524930 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3023cd94-3510-46bd-8aa4-0acc10b99634 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:42.526644 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb05929-5b69-4d65-b48b-c5d9be1a77ff admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:42.527101 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3023cd94-3510-46bd-8aa4-0acc10b99634 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:42.539155 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 911/1833] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:42.541566 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb05929-5b69-4d65-b48b-c5d9be1a77ff admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:42.551679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb05929-5b69-4d65-b48b-c5d9be1a77ff admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:42.556267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46123050-01a1-4488-8c6a-173c9aee8b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:42.556946 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46123050-01a1-4488-8c6a-173c9aee8b32 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:42.557227 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46123050-01a1-4488-8c6a-173c9aee8b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:42.557507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46123050-01a1-4488-8c6a-173c9aee8b32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:42.558227 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46123050-01a1-4488-8c6a-173c9aee8b32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:42.559814 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46123050-01a1-4488-8c6a-173c9aee8b32 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:42.569748 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 912/1834] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:07:42.573998 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46123050-01a1-4488-8c6a-173c9aee8b32 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:42.583824 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46123050-01a1-4488-8c6a-173c9aee8b32 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:42.591245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917b99c-bd96-4515-87ff-89adfceea8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:42.591862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917b99c-bd96-4515-87ff-89adfceea8fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:42.592146 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917b99c-bd96-4515-87ff-89adfceea8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:42.592455 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917b99c-bd96-4515-87ff-89adfceea8fb admin admin] PATH_INFO: `/v3/projects/ffc2ec84930f4bd9b5e7b53d82b1b1ec/users/a1520e93cfc5403a9f5e29f6cbe1ce14/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:42.609274 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917b99c-bd96-4515-87ff-89adfceea8fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffc2ec84930f4bd9b5e7b53d82b1b1ec, user_id=a1520e93cfc5403a9f5e29f6cbe1ce14, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:42.612974 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917b99c-bd96-4515-87ff-89adfceea8fb admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:42.662171 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 923/1835] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:07:42 2022] POST /identity/v3/users => generated 287 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:42.663186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023cd94-3510-46bd-8aa4-0acc10b99634 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:42.672918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023cd94-3510-46bd-8aa4-0acc10b99634 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:42.678175 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d401ea-85cd-4448-9927-86e3e48668c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:42.678833 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d401ea-85cd-4448-9927-86e3e48668c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:42.679116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d401ea-85cd-4448-9927-86e3e48668c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:42.679389 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d401ea-85cd-4448-9927-86e3e48668c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:42.680111 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d401ea-85cd-4448-9927-86e3e48668c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:42.681753 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d401ea-85cd-4448-9927-86e3e48668c7 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:42.697681 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 924/1836] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:07:42.702021 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d401ea-85cd-4448-9927-86e3e48668c7 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:42.719702 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 913/1837] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:42 2022] PUT /identity/v3/projects/ffc2ec84930f4bd9b5e7b53d82b1b1ec/users/a1520e93cfc5403a9f5e29f6cbe1ce14/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:07:42.719841 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d401ea-85cd-4448-9927-86e3e48668c7 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:42.732479 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bb552-6e18-405f-b730-8f4a2391a5fb None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:42.733179 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bb552-6e18-405f-b730-8f4a2391a5fb None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:42.733681 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6bb552-6e18-405f-b730-8f4a2391a5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:42.798442 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-af6bb552-6e18-405f-b730-8f4a2391a5fb None None] MFA Rules not processed for user `a1520e93cfc5403a9f5e29f6cbe1ce14`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:42.800607 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-af6bb552-6e18-405f-b730-8f4a2391a5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:42.880422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de132f42-6647-42ac-b31f-1eb6570ee021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:42.881836 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de132f42-6647-42ac-b31f-1eb6570ee021 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:42.882278 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de132f42-6647-42ac-b31f-1eb6570ee021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:42.882696 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de132f42-6647-42ac-b31f-1eb6570ee021 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:42.883582 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de132f42-6647-42ac-b31f-1eb6570ee021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:42.885939 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de132f42-6647-42ac-b31f-1eb6570ee021 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:42.901305 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 925/1838] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:07:42.905423 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de132f42-6647-42ac-b31f-1eb6570ee021 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:42.915891 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 914/1839] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:07:42.917221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6bb552-6e18-405f-b730-8f4a2391a5fb None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:42.917510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de132f42-6647-42ac-b31f-1eb6570ee021 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:42.922993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60f61a-1fc2-4d20-8388-147f4c593a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:42.924065 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60f61a-1fc2-4d20-8388-147f4c593a5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:42.924447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60f61a-1fc2-4d20-8388-147f4c593a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:42.924800 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60f61a-1fc2-4d20-8388-147f4c593a5f admin admin] PATH_INFO: `/v3/projects/b2094e240f6240a2a1eba16eb920d631/users/1beaf9ae0c0c47bf8c43da98cfd8aa1b/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:42.929601 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6bb552-6e18-405f-b730-8f4a2391a5fb None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:42.934144 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5415e15-1572-4a7a-ac24-bcb73aff5b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:42.934795 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5415e15-1572-4a7a-ac24-bcb73aff5b70 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:42.935074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5415e15-1572-4a7a-ac24-bcb73aff5b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:42.935344 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5415e15-1572-4a7a-ac24-bcb73aff5b70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:42.936165 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5415e15-1572-4a7a-ac24-bcb73aff5b70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:42.938813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5415e15-1572-4a7a-ac24-bcb73aff5b70 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:42.950617 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60f61a-1fc2-4d20-8388-147f4c593a5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2094e240f6240a2a1eba16eb920d631, user_id=1beaf9ae0c0c47bf8c43da98cfd8aa1b, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:42.953641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60f61a-1fc2-4d20-8388-147f4c593a5f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:43.099567 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 926/1840] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:42 2022] PUT /identity/v3/projects/b2094e240f6240a2a1eba16eb920d631/users/1beaf9ae0c0c47bf8c43da98cfd8aa1b/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:07:43.110021 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb449b6-63c8-45ee-8335-fe059460c4f3 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:43.110443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb449b6-63c8-45ee-8335-fe059460c4f3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:43.110791 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb449b6-63c8-45ee-8335-fe059460c4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:43.140874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-cbb449b6-63c8-45ee-8335-fe059460c4f3 None None] MFA Rules not processed for user `1beaf9ae0c0c47bf8c43da98cfd8aa1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:43.142170 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-cbb449b6-63c8-45ee-8335-fe059460c4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:43.159232 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:43.163930 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 915/1841] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:07:42 2022] POST /identity/v3/projects => generated 381 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:43.169379 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5415e15-1572-4a7a-ac24-bcb73aff5b70 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:43.267817 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5415e15-1572-4a7a-ac24-bcb73aff5b70 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:43.272791 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94812807-6bf0-4c09-9015-51d805f86e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:43.273478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94812807-6bf0-4c09-9015-51d805f86e2c admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:43.275311 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94812807-6bf0-4c09-9015-51d805f86e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:43.275605 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94812807-6bf0-4c09-9015-51d805f86e2c admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:43.276347 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94812807-6bf0-4c09-9015-51d805f86e2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:43.278343 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94812807-6bf0-4c09-9015-51d805f86e2c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:43.343877 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 927/1842] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:43.355401 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb449b6-63c8-45ee-8335-fe059460c4f3 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:43.356933 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-cbb449b6-63c8-45ee-8335-fe059460c4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:43.362253 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 916/1843] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:07:43 2022] POST /identity/v3/users => generated 297 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:43.365713 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94812807-6bf0-4c09-9015-51d805f86e2c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:43.383245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94812807-6bf0-4c09-9015-51d805f86e2c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:43.385614 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834c6f4-3251-4fa4-b301-587c1e2fbd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:43.386325 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834c6f4-3251-4fa4-b301-587c1e2fbd22 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:43.386641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834c6f4-3251-4fa4-b301-587c1e2fbd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:43.386913 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834c6f4-3251-4fa4-b301-587c1e2fbd22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:43.387650 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f834c6f4-3251-4fa4-b301-587c1e2fbd22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:43.389304 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f834c6f4-3251-4fa4-b301-587c1e2fbd22 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:43.405985 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 917/1844] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:07:43.409595 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834c6f4-3251-4fa4-b301-587c1e2fbd22 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:43.418033 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834c6f4-3251-4fa4-b301-587c1e2fbd22 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:43.422874 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b3fc5-71fa-4703-a41a-906760f35c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:43.424897 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b3fc5-71fa-4703-a41a-906760f35c42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:43.425215 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b3fc5-71fa-4703-a41a-906760f35c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:43.425494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b3fc5-71fa-4703-a41a-906760f35c42 admin admin] PATH_INFO: `/v3/projects/f96cd41ec9954bd986bbdee435254b5d/users/5868fa74600f42398cfc3449d8f0e0da/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:43.427871 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb449b6-63c8-45ee-8335-fe059460c4f3 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:43.443377 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b3fc5-71fa-4703-a41a-906760f35c42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96cd41ec9954bd986bbdee435254b5d, user_id=5868fa74600f42398cfc3449d8f0e0da, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:43.446233 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b3fc5-71fa-4703-a41a-906760f35c42 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:43.495178 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0a2a3-10cc-4090-817a-f2c45cd8397a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:43.495865 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0a2a3-10cc-4090-817a-f2c45cd8397a service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:43.496426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0a2a3-10cc-4090-817a-f2c45cd8397a service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:43.496710 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0a2a3-10cc-4090-817a-f2c45cd8397a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:43.544183 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 918/1845] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:43 2022] PUT /identity/v3/projects/f96cd41ec9954bd986bbdee435254b5d/users/5868fa74600f42398cfc3449d8f0e0da/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:43.545179 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b3fc5-71fa-4703-a41a-906760f35c42 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:43.577363 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b0a2a3-10cc-4090-817a-f2c45cd8397a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:43.579592 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b0a2a3-10cc-4090-817a-f2c45cd8397a service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:43.592848 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 928/1846] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:43.633338 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b3fc5-71fa-4703-a41a-906760f35c42 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:43.638042 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f501a-ed81-4a01-85c7-a281d77fa6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:43.638832 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f501a-ed81-4a01-85c7-a281d77fa6f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:43.639285 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f501a-ed81-4a01-85c7-a281d77fa6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:43.639634 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f501a-ed81-4a01-85c7-a281d77fa6f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:43.640490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f501a-ed81-4a01-85c7-a281d77fa6f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:43.642302 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f501a-ed81-4a01-85c7-a281d77fa6f0 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:43.652840 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 919/1847] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:43.657502 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0a2a3-10cc-4090-817a-f2c45cd8397a service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:43.668469 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0a2a3-10cc-4090-817a-f2c45cd8397a service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:43.684878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47d454-5986-4a7d-be99-b780e527d460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:43.684878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47d454-5986-4a7d-be99-b780e527d460 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:43.684878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47d454-5986-4a7d-be99-b780e527d460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:43.684878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47d454-5986-4a7d-be99-b780e527d460 admin admin] PATH_INFO: `/v3/projects/f96cd41ec9954bd986bbdee435254b5d/users/5868fa74600f42398cfc3449d8f0e0da/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:43.685353 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e47d454-5986-4a7d-be99-b780e527d460 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96cd41ec9954bd986bbdee435254b5d, user_id=5868fa74600f42398cfc3449d8f0e0da, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:43.691812 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e47d454-5986-4a7d-be99-b780e527d460 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:43.719493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 929/1848] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:43 2022] PUT /identity/v3/projects/f96cd41ec9954bd986bbdee435254b5d/users/5868fa74600f42398cfc3449d8f0e0da/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:43.734046 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada5ce8-d315-40f0-b24e-4504d958b6bf None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:43.734339 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada5ce8-d315-40f0-b24e-4504d958b6bf None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:43.734636 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada5ce8-d315-40f0-b24e-4504d958b6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:43.764407 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-eada5ce8-d315-40f0-b24e-4504d958b6bf None None] MFA Rules not processed for user `5868fa74600f42398cfc3449d8f0e0da`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:43.765685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-eada5ce8-d315-40f0-b24e-4504d958b6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:43.844540 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 920/1849] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:43 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:43.853597 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47d454-5986-4a7d-be99-b780e527d460 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:43.855878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7e47d454-5986-4a7d-be99-b780e527d460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:43.919396 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47d454-5986-4a7d-be99-b780e527d460 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:43.923508 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a28e479-f366-47ec-9535-cb3ae63cc44c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:43.924208 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28e479-f366-47ec-9535-cb3ae63cc44c service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:43.925183 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28e479-f366-47ec-9535-cb3ae63cc44c service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:43.925464 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28e479-f366-47ec-9535-cb3ae63cc44c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:43.935008 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a28e479-f366-47ec-9535-cb3ae63cc44c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:43.936544 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a28e479-f366-47ec-9535-cb3ae63cc44c service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:43.946511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 930/1850] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:43 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:44.105782 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:44.200624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada5ce8-d315-40f0-b24e-4504d958b6bf None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:44.210632 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada5ce8-d315-40f0-b24e-4504d958b6bf None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:44.214260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822828d-802d-491f-bd63-e5028087dee6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:44.214900 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822828d-802d-491f-bd63-e5028087dee6 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:44.215185 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822828d-802d-491f-bd63-e5028087dee6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:44.215458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822828d-802d-491f-bd63-e5028087dee6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:44.287609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822828d-802d-491f-bd63-e5028087dee6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:44.289453 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822828d-802d-491f-bd63-e5028087dee6 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:44.301103 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 921/1851] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:44.431106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:44.805408 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a28e479-f366-47ec-9535-cb3ae63cc44c service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:44.871458 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a28e479-f366-47ec-9535-cb3ae63cc44c service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:44.876236 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d2f28-9efc-4818-991c-740a1808526b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:44.877716 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d2f28-9efc-4818-991c-740a1808526b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:44.877998 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d2f28-9efc-4818-991c-740a1808526b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:44.878273 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d2f28-9efc-4818-991c-740a1808526b admin admin] PATH_INFO: `/v3/users/a1520e93cfc5403a9f5e29f6cbe1ce14` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:44.879513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d2f28-9efc-4818-991c-740a1808526b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1520e93cfc5403a9f5e29f6cbe1ce14)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:44.881492 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d2f28-9efc-4818-991c-740a1808526b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:45.252595 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-525d2f28-9efc-4818-991c-740a1808526b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1520e93cfc5403a9f5e29f6cbe1ce14'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:45.254025 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-525d2f28-9efc-4818-991c-740a1808526b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1520e93cfc5403a9f5e29f6cbe1ce14'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:45.349101 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-525d2f28-9efc-4818-991c-740a1808526b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1520e93cfc5403a9f5e29f6cbe1ce14'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:45.349981 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-525d2f28-9efc-4818-991c-740a1808526b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1520e93cfc5403a9f5e29f6cbe1ce14'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:45.363792 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-525d2f28-9efc-4818-991c-740a1808526b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1520e93cfc5403a9f5e29f6cbe1ce14'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:45.375842 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 931/1852] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:07:44 2022] DELETE /identity/v3/users/a1520e93cfc5403a9f5e29f6cbe1ce14 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:45.380482 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c7a6f-299b-4f4f-b1dc-d87b05bf8023 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:45.380979 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c7a6f-299b-4f4f-b1dc-d87b05bf8023 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:45.381069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c7a6f-299b-4f4f-b1dc-d87b05bf8023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:45.397986 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-953c7a6f-299b-4f4f-b1dc-d87b05bf8023 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:45.399291 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-953c7a6f-299b-4f4f-b1dc-d87b05bf8023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:45.458113 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:45.458908 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 922/1853] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:45.467967 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d2f28-9efc-4818-991c-740a1808526b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:45.469590 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-525d2f28-9efc-4818-991c-740a1808526b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:45.527153 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d2f28-9efc-4818-991c-740a1808526b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:45.530748 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdef152-f576-43af-8f56-719bfbb09b38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:45.531402 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdef152-f576-43af-8f56-719bfbb09b38 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:45.531682 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdef152-f576-43af-8f56-719bfbb09b38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:45.535795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdef152-f576-43af-8f56-719bfbb09b38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:45.541455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdef152-f576-43af-8f56-719bfbb09b38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:45.542960 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdef152-f576-43af-8f56-719bfbb09b38 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:45.553022 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 932/1854] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:45.841348 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:46.060330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c7a6f-299b-4f4f-b1dc-d87b05bf8023 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:46.062256 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-953c7a6f-299b-4f4f-b1dc-d87b05bf8023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:46.078174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c7a6f-299b-4f4f-b1dc-d87b05bf8023 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:46.085136 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048f2d4-30de-4f1c-86c6-085f238ec775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:46.086206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048f2d4-30de-4f1c-86c6-085f238ec775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:46.087674 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048f2d4-30de-4f1c-86c6-085f238ec775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:46.087674 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048f2d4-30de-4f1c-86c6-085f238ec775 admin admin] PATH_INFO: `/v3/projects/ffc2ec84930f4bd9b5e7b53d82b1b1ec` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:46.088928 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f048f2d4-30de-4f1c-86c6-085f238ec775 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffc2ec84930f4bd9b5e7b53d82b1b1ec)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:46.094360 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f048f2d4-30de-4f1c-86c6-085f238ec775 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:46.221517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f048f2d4-30de-4f1c-86c6-085f238ec775 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffc2ec84930f4bd9b5e7b53d82b1b1ec', 'request_id': 'req-f048f2d4-30de-4f1c-86c6-085f238ec775'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:46.228560 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f048f2d4-30de-4f1c-86c6-085f238ec775 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffc2ec84930f4bd9b5e7b53d82b1b1ec', 'request_id': 'req-f048f2d4-30de-4f1c-86c6-085f238ec775'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:46.238559 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f048f2d4-30de-4f1c-86c6-085f238ec775 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffc2ec84930f4bd9b5e7b53d82b1b1ec', 'request_id': 'req-f048f2d4-30de-4f1c-86c6-085f238ec775'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:46.351447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f048f2d4-30de-4f1c-86c6-085f238ec775 admin admin] The token cache is being invalidate because project ffc2ec84930f4bd9b5e7b53d82b1b1ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:07:46.351740 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f048f2d4-30de-4f1c-86c6-085f238ec775 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:46.352798 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f048f2d4-30de-4f1c-86c6-085f238ec775 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:46.356982 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 923/1855] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:07:46 2022] DELETE /identity/v3/projects/ffc2ec84930f4bd9b5e7b53d82b1b1ec => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:46.362506 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdef152-f576-43af-8f56-719bfbb09b38 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:46.374569 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdef152-f576-43af-8f56-719bfbb09b38 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:46.454574 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1160ded-339d-40df-be96-ec6632bba156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:46.455568 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1160ded-339d-40df-be96-ec6632bba156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:46.455860 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1160ded-339d-40df-be96-ec6632bba156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:46.456137 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1160ded-339d-40df-be96-ec6632bba156 admin admin] PATH_INFO: `/v3/users/5868fa74600f42398cfc3449d8f0e0da` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:46.457899 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1160ded-339d-40df-be96-ec6632bba156 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5868fa74600f42398cfc3449d8f0e0da)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:46.459871 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1160ded-339d-40df-be96-ec6632bba156 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:46.683636 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048f2d4-30de-4f1c-86c6-085f238ec775 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:46.685731 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f048f2d4-30de-4f1c-86c6-085f238ec775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:46.735967 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c1160ded-339d-40df-be96-ec6632bba156 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5868fa74600f42398cfc3449d8f0e0da'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:46.737893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c1160ded-339d-40df-be96-ec6632bba156 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5868fa74600f42398cfc3449d8f0e0da'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:46.746637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048f2d4-30de-4f1c-86c6-085f238ec775 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:46.752741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac51d5-3d7d-4805-a6bf-112e573fd9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:46.754059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac51d5-3d7d-4805-a6bf-112e573fd9a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:46.754572 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac51d5-3d7d-4805-a6bf-112e573fd9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:46.755296 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aac51d5-3d7d-4805-a6bf-112e573fd9a5 admin admin] PATH_INFO: `/v3/users/371f624f4ec141df9b439c89450176ea` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:46.757049 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac51d5-3d7d-4805-a6bf-112e573fd9a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=371f624f4ec141df9b439c89450176ea)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:46.763337 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:46.766937 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aac51d5-3d7d-4805-a6bf-112e573fd9a5 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:46.880703 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c1160ded-339d-40df-be96-ec6632bba156 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5868fa74600f42398cfc3449d8f0e0da'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:46.881431 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c1160ded-339d-40df-be96-ec6632bba156 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5868fa74600f42398cfc3449d8f0e0da'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:46.899427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c1160ded-339d-40df-be96-ec6632bba156 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5868fa74600f42398cfc3449d8f0e0da'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:46.916113 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 933/1856] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:07:46 2022] DELETE /identity/v3/users/5868fa74600f42398cfc3449d8f0e0da => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:46.991690 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3aac51d5-3d7d-4805-a6bf-112e573fd9a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '371f624f4ec141df9b439c89450176ea'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:46.995557 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3aac51d5-3d7d-4805-a6bf-112e573fd9a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '371f624f4ec141df9b439c89450176ea'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:47.030529 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3aac51d5-3d7d-4805-a6bf-112e573fd9a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '371f624f4ec141df9b439c89450176ea'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:47.059122 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:47.068120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3aac51d5-3d7d-4805-a6bf-112e573fd9a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '371f624f4ec141df9b439c89450176ea'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:47.195477 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3aac51d5-3d7d-4805-a6bf-112e573fd9a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '371f624f4ec141df9b439c89450176ea'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:47.208214 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 924/1857] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:07:46 2022] DELETE /identity/v3/users/371f624f4ec141df9b439c89450176ea => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:47.210453 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaba266-aa3d-479d-b420-e0af90146f18 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:47.210846 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaba266-aa3d-479d-b420-e0af90146f18 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:47.211188 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaba266-aa3d-479d-b420-e0af90146f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:47.237878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-0eaba266-aa3d-479d-b420-e0af90146f18 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:47.241587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0eaba266-aa3d-479d-b420-e0af90146f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:47.340672 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 934/1858] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:47.350940 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac51d5-3d7d-4805-a6bf-112e573fd9a5 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:47.352490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3aac51d5-3d7d-4805-a6bf-112e573fd9a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:47.412972 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aac51d5-3d7d-4805-a6bf-112e573fd9a5 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:47.417372 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e2822-42ad-409d-a1a9-ebc5bb30b0b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:47.418270 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e2822-42ad-409d-a1a9-ebc5bb30b0b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:47.418645 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e2822-42ad-409d-a1a9-ebc5bb30b0b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:47.419015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e2822-42ad-409d-a1a9-ebc5bb30b0b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:47.428753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e2822-42ad-409d-a1a9-ebc5bb30b0b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:47.430274 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e2822-42ad-409d-a1a9-ebc5bb30b0b8 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:47.440322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 925/1859] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:47.758009 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaba266-aa3d-479d-b420-e0af90146f18 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:47.760935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0eaba266-aa3d-479d-b420-e0af90146f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:47.788796 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaba266-aa3d-479d-b420-e0af90146f18 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:47.797335 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd1f64-a963-4a0a-b011-b92353965555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:47.797837 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd1f64-a963-4a0a-b011-b92353965555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:47.798356 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd1f64-a963-4a0a-b011-b92353965555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:47.799036 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd1f64-a963-4a0a-b011-b92353965555 admin admin] PATH_INFO: `/v3/projects/f96cd41ec9954bd986bbdee435254b5d` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:47.801929 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd1f64-a963-4a0a-b011-b92353965555 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f96cd41ec9954bd986bbdee435254b5d)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:47.805650 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd1f64-a963-4a0a-b011-b92353965555 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:47.886369 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:47.932639 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-17fd1f64-a963-4a0a-b011-b92353965555 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f96cd41ec9954bd986bbdee435254b5d', 'request_id': 'req-17fd1f64-a963-4a0a-b011-b92353965555'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:48.010006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e2822-42ad-409d-a1a9-ebc5bb30b0b8 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:48.011975 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-cd8e2822-42ad-409d-a1a9-ebc5bb30b0b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:48.028119 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e2822-42ad-409d-a1a9-ebc5bb30b0b8 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:48.036600 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-17fd1f64-a963-4a0a-b011-b92353965555 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f96cd41ec9954bd986bbdee435254b5d', 'request_id': 'req-17fd1f64-a963-4a0a-b011-b92353965555'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:48.043246 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-17fd1f64-a963-4a0a-b011-b92353965555 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f96cd41ec9954bd986bbdee435254b5d', 'request_id': 'req-17fd1f64-a963-4a0a-b011-b92353965555'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:48.055246 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-17fd1f64-a963-4a0a-b011-b92353965555 admin admin] The token cache is being invalidate because project f96cd41ec9954bd986bbdee435254b5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:07:48.055620 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-17fd1f64-a963-4a0a-b011-b92353965555 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:48.056370 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-17fd1f64-a963-4a0a-b011-b92353965555 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:48.058206 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 935/1860] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:07:47 2022] DELETE /identity/v3/projects/f96cd41ec9954bd986bbdee435254b5d => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:48.063318 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a08bd6-2674-4ed3-a255-d77d20945bb0 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:48.063731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a08bd6-2674-4ed3-a255-d77d20945bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:48.064983 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a08bd6-2674-4ed3-a255-d77d20945bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:48.084403 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-37a08bd6-2674-4ed3-a255-d77d20945bb0 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:48.085742 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-37a08bd6-2674-4ed3-a255-d77d20945bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:48.096742 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90314b6c-fc85-4d6b-a973-1c7aed448d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:48.097615 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90314b6c-fc85-4d6b-a973-1c7aed448d47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:48.097975 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90314b6c-fc85-4d6b-a973-1c7aed448d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:48.098164 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90314b6c-fc85-4d6b-a973-1c7aed448d47 admin admin] PATH_INFO: `/v3/projects/bc4fa9ef01ed40208899eab624dbfc6a` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:48.099385 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90314b6c-fc85-4d6b-a973-1c7aed448d47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc4fa9ef01ed40208899eab624dbfc6a)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:48.101423 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90314b6c-fc85-4d6b-a973-1c7aed448d47 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:48.186538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 936/1861] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:07:48.194036 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2f73e-9016-45d8-8301-38e9be3cdb58 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:48.194402 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2f73e-9016-45d8-8301-38e9be3cdb58 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:48.194718 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2f73e-9016-45d8-8301-38e9be3cdb58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:48.219003 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-96f2f73e-9016-45d8-8301-38e9be3cdb58 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:48.219003 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-96f2f73e-9016-45d8-8301-38e9be3cdb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:48.233462 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 937/1862] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:07:48.237333 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f2f73e-9016-45d8-8301-38e9be3cdb58 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:48.247054 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f2f73e-9016-45d8-8301-38e9be3cdb58 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:48.255811 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee46e9-0824-4211-bbda-99241f8e3c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:48.256566 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee46e9-0824-4211-bbda-99241f8e3c76 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:48.256851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee46e9-0824-4211-bbda-99241f8e3c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:48.257122 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee46e9-0824-4211-bbda-99241f8e3c76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:48.257856 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ee46e9-0824-4211-bbda-99241f8e3c76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:07:48.258160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ee46e9-0824-4211-bbda-99241f8e3c76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:48.259812 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ee46e9-0824-4211-bbda-99241f8e3c76 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:48.274841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:48.276335 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 938/1863] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:07:48.280437 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee46e9-0824-4211-bbda-99241f8e3c76 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:48.295503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee46e9-0824-4211-bbda-99241f8e3c76 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:48.297367 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-90314b6c-fc85-4d6b-a973-1c7aed448d47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc4fa9ef01ed40208899eab624dbfc6a', 'request_id': 'req-90314b6c-fc85-4d6b-a973-1c7aed448d47'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:48.305120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-90314b6c-fc85-4d6b-a973-1c7aed448d47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc4fa9ef01ed40208899eab624dbfc6a', 'request_id': 'req-90314b6c-fc85-4d6b-a973-1c7aed448d47'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:48.313369 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-90314b6c-fc85-4d6b-a973-1c7aed448d47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc4fa9ef01ed40208899eab624dbfc6a', 'request_id': 'req-90314b6c-fc85-4d6b-a973-1c7aed448d47'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:48.360269 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8604a2-25fd-4dae-8d4e-ea60401a76f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:48.360992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8604a2-25fd-4dae-8d4e-ea60401a76f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:48.361297 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8604a2-25fd-4dae-8d4e-ea60401a76f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:48.361575 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8604a2-25fd-4dae-8d4e-ea60401a76f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:48.362267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8604a2-25fd-4dae-8d4e-ea60401a76f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:48.364187 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8604a2-25fd-4dae-8d4e-ea60401a76f3 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:48.395988 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-90314b6c-fc85-4d6b-a973-1c7aed448d47 admin admin] The token cache is being invalidate because project bc4fa9ef01ed40208899eab624dbfc6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:07:48.396287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-90314b6c-fc85-4d6b-a973-1c7aed448d47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:48.396979 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-90314b6c-fc85-4d6b-a973-1c7aed448d47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:48.398529 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 926/1864] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:07:48 2022] DELETE /identity/v3/projects/bc4fa9ef01ed40208899eab624dbfc6a => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:48.402372 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90314b6c-fc85-4d6b-a973-1c7aed448d47 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:48.403758 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-90314b6c-fc85-4d6b-a973-1c7aed448d47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:48.420013 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90314b6c-fc85-4d6b-a973-1c7aed448d47 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:48.425529 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed99ef5-ce41-4d56-814a-edc1d1519d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:48.426401 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed99ef5-ce41-4d56-814a-edc1d1519d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:48.426681 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed99ef5-ce41-4d56-814a-edc1d1519d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:48.426958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed99ef5-ce41-4d56-814a-edc1d1519d40 admin admin] PATH_INFO: `/v3/users/daee89a501184d919325f779fff8c8b4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:48.428164 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed99ef5-ce41-4d56-814a-edc1d1519d40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=daee89a501184d919325f779fff8c8b4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:48.430123 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed99ef5-ce41-4d56-814a-edc1d1519d40 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:48.430886 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 939/1865] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:07:48 2022] POST /identity/v3/projects => generated 373 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:07:48.435157 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8604a2-25fd-4dae-8d4e-ea60401a76f3 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:48.437167 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1d8604a2-25fd-4dae-8d4e-ea60401a76f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:48.499948 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8604a2-25fd-4dae-8d4e-ea60401a76f3 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:48.505172 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6624e-5049-4802-ad03-dc8699d44daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:48.505844 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6624e-5049-4802-ad03-dc8699d44daf admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:48.506124 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6624e-5049-4802-ad03-dc8699d44daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:48.506395 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6624e-5049-4802-ad03-dc8699d44daf admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:48.507079 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6624e-5049-4802-ad03-dc8699d44daf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:48.509092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd6624e-5049-4802-ad03-dc8699d44daf admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:48.609649 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 940/1866] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:07:48 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:48.609649 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6624e-5049-4802-ad03-dc8699d44daf admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:48.609649 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5fd6624e-5049-4802-ad03-dc8699d44daf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:48.622232 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6624e-5049-4802-ad03-dc8699d44daf admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:48.628049 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3bc5cf-3446-422b-8b99-41675747a1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:48.629114 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3bc5cf-3446-422b-8b99-41675747a1ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:48.629368 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3bc5cf-3446-422b-8b99-41675747a1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:48.629634 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3bc5cf-3446-422b-8b99-41675747a1ac admin admin] PATH_INFO: `/v3/users/1beaf9ae0c0c47bf8c43da98cfd8aa1b` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:48.630826 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3bc5cf-3446-422b-8b99-41675747a1ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=1beaf9ae0c0c47bf8c43da98cfd8aa1b)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:48.632990 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3bc5cf-3446-422b-8b99-41675747a1ac admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:48.689940 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9ed99ef5-ce41-4d56-814a-edc1d1519d40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daee89a501184d919325f779fff8c8b4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:48.690595 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9ed99ef5-ce41-4d56-814a-edc1d1519d40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daee89a501184d919325f779fff8c8b4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:48.702846 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9ed99ef5-ce41-4d56-814a-edc1d1519d40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daee89a501184d919325f779fff8c8b4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:48.710258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9ed99ef5-ce41-4d56-814a-edc1d1519d40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daee89a501184d919325f779fff8c8b4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:48.823366 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:48.823895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9ed99ef5-ce41-4d56-814a-edc1d1519d40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daee89a501184d919325f779fff8c8b4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:48.831320 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 927/1867] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:07:48 2022] DELETE /identity/v3/users/daee89a501184d919325f779fff8c8b4 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:48.832070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed99ef5-ce41-4d56-814a-edc1d1519d40 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:48.833716 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9ed99ef5-ce41-4d56-814a-edc1d1519d40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:48.878332 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1d3bc5cf-3446-422b-8b99-41675747a1ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1beaf9ae0c0c47bf8c43da98cfd8aa1b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:48.879236 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1d3bc5cf-3446-422b-8b99-41675747a1ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1beaf9ae0c0c47bf8c43da98cfd8aa1b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:48.928907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed99ef5-ce41-4d56-814a-edc1d1519d40 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:48.933363 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1fa1e9-d966-469e-90a4-df1ed47a882f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:48.934029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fa1e9-d966-469e-90a4-df1ed47a882f admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:48.934309 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fa1e9-d966-469e-90a4-df1ed47a882f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:48.934588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fa1e9-d966-469e-90a4-df1ed47a882f admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:48.935321 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1fa1e9-d966-469e-90a4-df1ed47a882f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:48.936959 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1fa1e9-d966-469e-90a4-df1ed47a882f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:48.947773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 928/1868] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:48.951727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1fa1e9-d966-469e-90a4-df1ed47a882f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:48.961279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1fa1e9-d966-469e-90a4-df1ed47a882f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:48.966098 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89abe6d1-9a47-4c89-8e64-9e5265fd6c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:48.966259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1d3bc5cf-3446-422b-8b99-41675747a1ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1beaf9ae0c0c47bf8c43da98cfd8aa1b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:48.966726 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1d3bc5cf-3446-422b-8b99-41675747a1ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1beaf9ae0c0c47bf8c43da98cfd8aa1b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:48.966848 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abe6d1-9a47-4c89-8e64-9e5265fd6c00 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:48.967013 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abe6d1-9a47-4c89-8e64-9e5265fd6c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:48.967281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abe6d1-9a47-4c89-8e64-9e5265fd6c00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:48.967994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89abe6d1-9a47-4c89-8e64-9e5265fd6c00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:48.969783 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89abe6d1-9a47-4c89-8e64-9e5265fd6c00 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:48.979542 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1d3bc5cf-3446-422b-8b99-41675747a1ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1beaf9ae0c0c47bf8c43da98cfd8aa1b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:48.980157 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 929/1869] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:07:48.984102 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89abe6d1-9a47-4c89-8e64-9e5265fd6c00 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:48.986704 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-89abe6d1-9a47-4c89-8e64-9e5265fd6c00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:49.002721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89abe6d1-9a47-4c89-8e64-9e5265fd6c00 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:49.003574 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 941/1870] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:07:48 2022] DELETE /identity/v3/users/1beaf9ae0c0c47bf8c43da98cfd8aa1b => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:49.008210 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c1096-ec43-44ac-a22b-023254a3d654 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:49.008584 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c1096-ec43-44ac-a22b-023254a3d654 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:49.008863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c1096-ec43-44ac-a22b-023254a3d654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:49.029106 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c33c1096-ec43-44ac-a22b-023254a3d654 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:49.030640 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c33c1096-ec43-44ac-a22b-023254a3d654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:49.065132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be26dc-c75a-4f1f-a22b-1505d6b250a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:49.066092 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be26dc-c75a-4f1f-a22b-1505d6b250a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:49.066376 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be26dc-c75a-4f1f-a22b-1505d6b250a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:49.066653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be26dc-c75a-4f1f-a22b-1505d6b250a9 admin admin] PATH_INFO: `/v3/projects/ce4e3f14283b45619321876b00b4d466/users/c41d51884a3b47318c6d05fba5918fa7/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:49.084274 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be26dc-c75a-4f1f-a22b-1505d6b250a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce4e3f14283b45619321876b00b4d466, user_id=c41d51884a3b47318c6d05fba5918fa7, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:49.087193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be26dc-c75a-4f1f-a22b-1505d6b250a9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:49.109339 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 942/1871] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:07:49.116582 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c1096-ec43-44ac-a22b-023254a3d654 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:49.117952 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c33c1096-ec43-44ac-a22b-023254a3d654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:49.150198 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 930/1872] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:48 2022] PUT /identity/v3/projects/ce4e3f14283b45619321876b00b4d466/users/c41d51884a3b47318c6d05fba5918fa7/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:07:49.155071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63515b-1f8a-4176-8e93-fb157fc75227 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:49.155361 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63515b-1f8a-4176-8e93-fb157fc75227 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:49.155636 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63515b-1f8a-4176-8e93-fb157fc75227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:49.173224 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-0e63515b-1f8a-4176-8e93-fb157fc75227 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:49.174987 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0e63515b-1f8a-4176-8e93-fb157fc75227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:49.181116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c1096-ec43-44ac-a22b-023254a3d654 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:49.231240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5366c64-9388-4ae3-8dc6-904224ad441e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:49.231905 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5366c64-9388-4ae3-8dc6-904224ad441e service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:49.232172 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5366c64-9388-4ae3-8dc6-904224ad441e service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:49.232473 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5366c64-9388-4ae3-8dc6-904224ad441e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:49.239343 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 931/1873] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:07:49.246554 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5366c64-9388-4ae3-8dc6-904224ad441e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:49.246554 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5366c64-9388-4ae3-8dc6-904224ad441e service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:49.246734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63515b-1f8a-4176-8e93-fb157fc75227 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:49.255687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 943/1874] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 20:07:49.256124 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63515b-1f8a-4176-8e93-fb157fc75227 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:49.260013 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf501e-8b73-4021-82f6-530d2a9245a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:49.260648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf501e-8b73-4021-82f6-530d2a9245a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:49.260933 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf501e-8b73-4021-82f6-530d2a9245a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:49.261209 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf501e-8b73-4021-82f6-530d2a9245a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:49.276101 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf501e-8b73-4021-82f6-530d2a9245a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:49.277800 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf501e-8b73-4021-82f6-530d2a9245a2 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:49.288539 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 932/1875] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 20:07:49.314602 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5366c64-9388-4ae3-8dc6-904224ad441e service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:49.316031 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f5366c64-9388-4ae3-8dc6-904224ad441e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:49.330269 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5366c64-9388-4ae3-8dc6-904224ad441e service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:49.335556 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93f5c4-a6c0-44e6-982c-26c7249365f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:49.336418 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93f5c4-a6c0-44e6-982c-26c7249365f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:49.336706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93f5c4-a6c0-44e6-982c-26c7249365f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:49.337059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93f5c4-a6c0-44e6-982c-26c7249365f9 admin admin] PATH_INFO: `/v3/projects/5ae70fd9426f415ea2cca344f8b9d4d8` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:49.338280 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf93f5c4-a6c0-44e6-982c-26c7249365f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ae70fd9426f415ea2cca344f8b9d4d8)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:49.340259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf93f5c4-a6c0-44e6-982c-26c7249365f9 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:49.387289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7442b-6c35-4110-9973-2369ffd0c795 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:49.387587 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7442b-6c35-4110-9973-2369ffd0c795 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:49.387857 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7442b-6c35-4110-9973-2369ffd0c795 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:49.390593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 933/1876] 10.222.0.44 () {54 vars in 921 bytes} [Thu Jun 23 20:07:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:07:49.405566 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbdd6b-cacf-425a-816f-1a817a00e409 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:49.405566 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbdd6b-cacf-425a-816f-1a817a00e409 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:49.405566 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbdd6b-cacf-425a-816f-1a817a00e409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:49.435904 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ebcbdd6b-cacf-425a-816f-1a817a00e409 None None] MFA Rules not processed for user `a63d1d902c024c97a38f7d92f73814d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:49.437172 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ebcbdd6b-cacf-425a-816f-1a817a00e409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:49.514259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-bf93f5c4-a6c0-44e6-982c-26c7249365f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ae70fd9426f415ea2cca344f8b9d4d8', 'request_id': 'req-bf93f5c4-a6c0-44e6-982c-26c7249365f9'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:49.529538 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 934/1877] 10.222.0.44 () {58 vars in 1018 bytes} [Thu Jun 23 20:07:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:07:49.583499 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-bf93f5c4-a6c0-44e6-982c-26c7249365f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ae70fd9426f415ea2cca344f8b9d4d8', 'request_id': 'req-bf93f5c4-a6c0-44e6-982c-26c7249365f9'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:49.589619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-bf93f5c4-a6c0-44e6-982c-26c7249365f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ae70fd9426f415ea2cca344f8b9d4d8', 'request_id': 'req-bf93f5c4-a6c0-44e6-982c-26c7249365f9'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:49.601558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-bf93f5c4-a6c0-44e6-982c-26c7249365f9 admin admin] The token cache is being invalidate because project 5ae70fd9426f415ea2cca344f8b9d4d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:07:49.601997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-bf93f5c4-a6c0-44e6-982c-26c7249365f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:49.602736 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-bf93f5c4-a6c0-44e6-982c-26c7249365f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:49.604411 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 944/1878] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:07:49 2022] DELETE /identity/v3/projects/5ae70fd9426f415ea2cca344f8b9d4d8 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:49.608361 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbdd6b-cacf-425a-816f-1a817a00e409 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:49.619228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbdd6b-cacf-425a-816f-1a817a00e409 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:49.694779 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77fcc3e-b3f1-4cfa-be2e-be06db9e2ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:49.695872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fcc3e-b3f1-4cfa-be2e-be06db9e2ae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:49.696154 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fcc3e-b3f1-4cfa-be2e-be06db9e2ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:49.696892 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fcc3e-b3f1-4cfa-be2e-be06db9e2ae1 admin admin] PATH_INFO: `/v3/users/cd0f2798ace54ccfb94a78080ddc2afe` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:49.699757 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77fcc3e-b3f1-4cfa-be2e-be06db9e2ae1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd0f2798ace54ccfb94a78080ddc2afe)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:49.703833 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77fcc3e-b3f1-4cfa-be2e-be06db9e2ae1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:49.749766 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:49.881272 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f77fcc3e-b3f1-4cfa-be2e-be06db9e2ae1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd0f2798ace54ccfb94a78080ddc2afe'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:49.882233 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f77fcc3e-b3f1-4cfa-be2e-be06db9e2ae1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd0f2798ace54ccfb94a78080ddc2afe'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:49.896624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f77fcc3e-b3f1-4cfa-be2e-be06db9e2ae1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd0f2798ace54ccfb94a78080ddc2afe'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:49.903940 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f77fcc3e-b3f1-4cfa-be2e-be06db9e2ae1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd0f2798ace54ccfb94a78080ddc2afe'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:49.977077 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f77fcc3e-b3f1-4cfa-be2e-be06db9e2ae1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd0f2798ace54ccfb94a78080ddc2afe'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:49.985797 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 935/1879] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:07:49 2022] DELETE /identity/v3/users/cd0f2798ace54ccfb94a78080ddc2afe => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:50.178533 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93f5c4-a6c0-44e6-982c-26c7249365f9 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:50.181864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bf93f5c4-a6c0-44e6-982c-26c7249365f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:50.276259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93f5c4-a6c0-44e6-982c-26c7249365f9 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:50.372044 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be07653-e5d0-4430-99c1-4dbde4898210 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:50.372726 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be07653-e5d0-4430-99c1-4dbde4898210 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:50.373006 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be07653-e5d0-4430-99c1-4dbde4898210 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:50.373279 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be07653-e5d0-4430-99c1-4dbde4898210 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:50.375079 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4be07653-e5d0-4430-99c1-4dbde4898210 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:50.386906 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be07653-e5d0-4430-99c1-4dbde4898210 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:50.388624 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be07653-e5d0-4430-99c1-4dbde4898210 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:50.399978 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 945/1880] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:50.420149 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77fcc3e-b3f1-4cfa-be2e-be06db9e2ae1 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:50.421753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f77fcc3e-b3f1-4cfa-be2e-be06db9e2ae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:50.544564 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77fcc3e-b3f1-4cfa-be2e-be06db9e2ae1 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:50.549427 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caa66bb-e1df-452e-85af-027e08a26877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:50.550333 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa66bb-e1df-452e-85af-027e08a26877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:50.550620 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa66bb-e1df-452e-85af-027e08a26877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:50.551037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caa66bb-e1df-452e-85af-027e08a26877 admin admin] PATH_INFO: `/v3/projects/ca9ee084981d46afa36ba6c90b84a2f9` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:50.552251 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caa66bb-e1df-452e-85af-027e08a26877 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca9ee084981d46afa36ba6c90b84a2f9)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:50.554350 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caa66bb-e1df-452e-85af-027e08a26877 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:50.728911 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2caa66bb-e1df-452e-85af-027e08a26877 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca9ee084981d46afa36ba6c90b84a2f9', 'request_id': 'req-2caa66bb-e1df-452e-85af-027e08a26877'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:50.736931 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2caa66bb-e1df-452e-85af-027e08a26877 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca9ee084981d46afa36ba6c90b84a2f9', 'request_id': 'req-2caa66bb-e1df-452e-85af-027e08a26877'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:50.743340 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2caa66bb-e1df-452e-85af-027e08a26877 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca9ee084981d46afa36ba6c90b84a2f9', 'request_id': 'req-2caa66bb-e1df-452e-85af-027e08a26877'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:50.881517 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:50.894287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2caa66bb-e1df-452e-85af-027e08a26877 admin admin] The token cache is being invalidate because project ca9ee084981d46afa36ba6c90b84a2f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:07:50.894539 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2caa66bb-e1df-452e-85af-027e08a26877 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:50.900549 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2caa66bb-e1df-452e-85af-027e08a26877 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:50.906572 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 936/1881] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:07:50 2022] DELETE /identity/v3/projects/ca9ee084981d46afa36ba6c90b84a2f9 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:50.908372 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fef39c-eaa1-49f7-a799-c0bac63cd4e0 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:50.913255 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fef39c-eaa1-49f7-a799-c0bac63cd4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:50.913255 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fef39c-eaa1-49f7-a799-c0bac63cd4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:50.927849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b2fef39c-eaa1-49f7-a799-c0bac63cd4e0 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:50.929090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b2fef39c-eaa1-49f7-a799-c0bac63cd4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:51.000515 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 946/1882] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:51.005362 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8b984-ff5e-4b67-bbcd-138ff01b33a8 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:51.005674 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8b984-ff5e-4b67-bbcd-138ff01b33a8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:51.005915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8b984-ff5e-4b67-bbcd-138ff01b33a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:51.034401 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-11d8b984-ff5e-4b67-bbcd-138ff01b33a8 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:51.035636 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-11d8b984-ff5e-4b67-bbcd-138ff01b33a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:51.047568 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 937/1883] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:51.049442 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fef39c-eaa1-49f7-a799-c0bac63cd4e0 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:51.061786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fef39c-eaa1-49f7-a799-c0bac63cd4e0 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:51.067894 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b9e05-9da2-4549-aaeb-409cf0862407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:51.068685 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b9e05-9da2-4549-aaeb-409cf0862407 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:51.068971 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b9e05-9da2-4549-aaeb-409cf0862407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:51.069245 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b9e05-9da2-4549-aaeb-409cf0862407 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:51.070119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b9e05-9da2-4549-aaeb-409cf0862407 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:07:51.070511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b9e05-9da2-4549-aaeb-409cf0862407 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:51.072514 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b9e05-9da2-4549-aaeb-409cf0862407 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:51.086214 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 947/1884] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:07:51.089981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8b984-ff5e-4b67-bbcd-138ff01b33a8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:51.103291 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8b984-ff5e-4b67-bbcd-138ff01b33a8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:51.105615 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe0941-6947-453a-b260-4940aecd1fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:51.106383 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe0941-6947-453a-b260-4940aecd1fcb admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:51.106764 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe0941-6947-453a-b260-4940aecd1fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:51.107136 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe0941-6947-453a-b260-4940aecd1fcb admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:51.108120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe0941-6947-453a-b260-4940aecd1fcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:51.110359 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe0941-6947-453a-b260-4940aecd1fcb admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:51.220997 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 938/1885] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:07:51 2022] POST /identity/v3/projects => generated 393 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:51.222282 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b9e05-9da2-4549-aaeb-409cf0862407 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:51.237163 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b9e05-9da2-4549-aaeb-409cf0862407 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:51.313089 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa9b40-5deb-4685-850e-f9d1581ad80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:51.313906 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa9b40-5deb-4685-850e-f9d1581ad80a admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:51.314478 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa9b40-5deb-4685-850e-f9d1581ad80a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:51.314869 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa9b40-5deb-4685-850e-f9d1581ad80a admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:51.315846 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa9b40-5deb-4685-850e-f9d1581ad80a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:51.317977 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efa9b40-5deb-4685-850e-f9d1581ad80a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:51.468014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 948/1886] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:07:51 2022] POST /identity/v3/users => generated 297 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:51.469424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe0941-6947-453a-b260-4940aecd1fcb admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:51.481528 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe0941-6947-453a-b260-4940aecd1fcb admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:51.487840 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1073ae69-1730-421e-ba3e-a77972c93ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:51.488916 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1073ae69-1730-421e-ba3e-a77972c93ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:51.490416 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1073ae69-1730-421e-ba3e-a77972c93ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:51.490416 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1073ae69-1730-421e-ba3e-a77972c93ef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:51.491126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1073ae69-1730-421e-ba3e-a77972c93ef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:51.493162 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1073ae69-1730-421e-ba3e-a77972c93ef8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:51.505847 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 939/1887] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:51.515235 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa9b40-5deb-4685-850e-f9d1581ad80a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:51.521133 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1073ae69-1730-421e-ba3e-a77972c93ef8 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:51.523294 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1073ae69-1730-421e-ba3e-a77972c93ef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:51.533869 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa9b40-5deb-4685-850e-f9d1581ad80a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:51.540635 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac0050-a09f-41a4-bcaa-7b3571727465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:51.541315 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac0050-a09f-41a4-bcaa-7b3571727465 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:51.541590 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac0050-a09f-41a4-bcaa-7b3571727465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:51.541862 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac0050-a09f-41a4-bcaa-7b3571727465 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:51.542590 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac0050-a09f-41a4-bcaa-7b3571727465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:51.542857 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1073ae69-1730-421e-ba3e-a77972c93ef8 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:51.544291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac0050-a09f-41a4-bcaa-7b3571727465 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:51.547662 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376c71a-b419-41c2-a168-b648567d5c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:51.549283 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376c71a-b419-41c2-a168-b648567d5c09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:51.549508 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376c71a-b419-41c2-a168-b648567d5c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:51.549783 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376c71a-b419-41c2-a168-b648567d5c09 admin admin] PATH_INFO: `/v3/projects/b2094e240f6240a2a1eba16eb920d631` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:51.551219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5376c71a-b419-41c2-a168-b648567d5c09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2094e240f6240a2a1eba16eb920d631)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:51.553389 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5376c71a-b419-41c2-a168-b648567d5c09 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:51.555930 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 949/1888] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:51.559981 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac0050-a09f-41a4-bcaa-7b3571727465 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:51.570984 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac0050-a09f-41a4-bcaa-7b3571727465 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:51.576572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc710e84-50b8-4fe4-8c5d-586a8ed9e007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:51.577452 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc710e84-50b8-4fe4-8c5d-586a8ed9e007 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:51.578262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc710e84-50b8-4fe4-8c5d-586a8ed9e007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:51.578392 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc710e84-50b8-4fe4-8c5d-586a8ed9e007 admin admin] PATH_INFO: `/v3/projects/2658c0810091420b9898887ba4cf7288/users/65ec664c077445ad83d93c54db88b6ce/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:51.593474 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc710e84-50b8-4fe4-8c5d-586a8ed9e007 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2658c0810091420b9898887ba4cf7288, user_id=65ec664c077445ad83d93c54db88b6ce, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:51.596747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc710e84-50b8-4fe4-8c5d-586a8ed9e007 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:51.734741 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 950/1889] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:51 2022] PUT /identity/v3/projects/2658c0810091420b9898887ba4cf7288/users/65ec664c077445ad83d93c54db88b6ce/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:07:51.747314 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff21d4a-dca9-4cfe-b856-6e8c732c34ba None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:51.747925 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff21d4a-dca9-4cfe-b856-6e8c732c34ba None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:51.750080 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff21d4a-dca9-4cfe-b856-6e8c732c34ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:51.767860 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5376c71a-b419-41c2-a168-b648567d5c09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2094e240f6240a2a1eba16eb920d631', 'request_id': 'req-5376c71a-b419-41c2-a168-b648567d5c09'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:51.780603 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5376c71a-b419-41c2-a168-b648567d5c09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2094e240f6240a2a1eba16eb920d631', 'request_id': 'req-5376c71a-b419-41c2-a168-b648567d5c09'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:51.814636 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3ff21d4a-dca9-4cfe-b856-6e8c732c34ba None None] MFA Rules not processed for user `65ec664c077445ad83d93c54db88b6ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:51.816226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3ff21d4a-dca9-4cfe-b856-6e8c732c34ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:51.820173 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5376c71a-b419-41c2-a168-b648567d5c09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2094e240f6240a2a1eba16eb920d631', 'request_id': 'req-5376c71a-b419-41c2-a168-b648567d5c09'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:51.896477 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 951/1890] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:07:51.901233 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff21d4a-dca9-4cfe-b856-6e8c732c34ba None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:51.945598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5376c71a-b419-41c2-a168-b648567d5c09 admin admin] The token cache is being invalidate because project b2094e240f6240a2a1eba16eb920d631 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:07:51.945892 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5376c71a-b419-41c2-a168-b648567d5c09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:51.952258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5376c71a-b419-41c2-a168-b648567d5c09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:07:51.957825 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 940/1891] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:07:51 2022] DELETE /identity/v3/projects/b2094e240f6240a2a1eba16eb920d631 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:51.966138 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fad364-fb64-4733-99d6-55afd2a9e852 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:51.966437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fad364-fb64-4733-99d6-55afd2a9e852 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:51.966718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fad364-fb64-4733-99d6-55afd2a9e852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:51.972989 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:51.990062 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff21d4a-dca9-4cfe-b856-6e8c732c34ba None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:51.995293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7d188-3bc2-4eb5-8f89-1f54ea8b2105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:51.998868 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7d188-3bc2-4eb5-8f89-1f54ea8b2105 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:51.998868 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7d188-3bc2-4eb5-8f89-1f54ea8b2105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:51.998868 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7d188-3bc2-4eb5-8f89-1f54ea8b2105 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:51.998868 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7d188-3bc2-4eb5-8f89-1f54ea8b2105 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:51.999757 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7d188-3bc2-4eb5-8f89-1f54ea8b2105 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:52.018827 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f0fad364-fb64-4733-99d6-55afd2a9e852 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:52.020488 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f0fad364-fb64-4733-99d6-55afd2a9e852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:52.040165 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 941/1892] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:52.047689 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7010c8-6ff9-4180-81d1-c104ed0dae7e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:52.047689 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7010c8-6ff9-4180-81d1-c104ed0dae7e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:52.047689 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7010c8-6ff9-4180-81d1-c104ed0dae7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:52.068836 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6a7010c8-6ff9-4180-81d1-c104ed0dae7e None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:52.071929 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6a7010c8-6ff9-4180-81d1-c104ed0dae7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:52.087987 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 942/1893] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:52.089211 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7010c8-6ff9-4180-81d1-c104ed0dae7e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:52.099237 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7010c8-6ff9-4180-81d1-c104ed0dae7e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:52.104026 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b5a53-69d5-4a7c-a7f5-340000505e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:52.104809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b5a53-69d5-4a7c-a7f5-340000505e74 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:52.105090 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b5a53-69d5-4a7c-a7f5-340000505e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:52.105391 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b5a53-69d5-4a7c-a7f5-340000505e74 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:52.106128 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b5a53-69d5-4a7c-a7f5-340000505e74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:07:52.106440 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b5a53-69d5-4a7c-a7f5-340000505e74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:52.108245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b5a53-69d5-4a7c-a7f5-340000505e74 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:52.122042 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 943/1894] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:07:52.129124 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b5a53-69d5-4a7c-a7f5-340000505e74 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:52.139623 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b5a53-69d5-4a7c-a7f5-340000505e74 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:52.145372 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5891a2f8-6ef6-4a06-8ba0-6bbda644b8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:52.146149 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891a2f8-6ef6-4a06-8ba0-6bbda644b8ab admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:52.146537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891a2f8-6ef6-4a06-8ba0-6bbda644b8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:52.146939 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891a2f8-6ef6-4a06-8ba0-6bbda644b8ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:52.147800 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5891a2f8-6ef6-4a06-8ba0-6bbda644b8ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:52.151416 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 952/1895] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:07:51 2022] POST /identity/v3/projects => generated 391 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:52.151563 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5891a2f8-6ef6-4a06-8ba0-6bbda644b8ab admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:52.152612 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7d188-3bc2-4eb5-8f89-1f54ea8b2105 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:52.154626 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-61b7d188-3bc2-4eb5-8f89-1f54ea8b2105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:52.241470 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7d188-3bc2-4eb5-8f89-1f54ea8b2105 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:52.246383 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07061bbd-2aef-47e9-abb0-89e4cc60569f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:52.247111 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07061bbd-2aef-47e9-abb0-89e4cc60569f admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:52.247332 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07061bbd-2aef-47e9-abb0-89e4cc60569f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:52.247633 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07061bbd-2aef-47e9-abb0-89e4cc60569f admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:52.248341 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07061bbd-2aef-47e9-abb0-89e4cc60569f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:52.250489 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07061bbd-2aef-47e9-abb0-89e4cc60569f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:52.301063 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 944/1896] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:07:52 2022] POST /identity/v3/projects => generated 379 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:52.305479 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5891a2f8-6ef6-4a06-8ba0-6bbda644b8ab admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:52.384449 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5891a2f8-6ef6-4a06-8ba0-6bbda644b8ab admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:52.389698 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31ab1f-49d5-436b-ab80-05f85c08aae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:52.390377 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31ab1f-49d5-436b-ab80-05f85c08aae8 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:52.390678 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31ab1f-49d5-436b-ab80-05f85c08aae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:52.390956 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31ab1f-49d5-436b-ab80-05f85c08aae8 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:52.391664 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f31ab1f-49d5-436b-ab80-05f85c08aae8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:52.393772 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f31ab1f-49d5-436b-ab80-05f85c08aae8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:52.408710 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 953/1897] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:07:52 2022] POST /identity/v3/users => generated 302 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:52.410062 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07061bbd-2aef-47e9-abb0-89e4cc60569f admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:52.421471 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07061bbd-2aef-47e9-abb0-89e4cc60569f admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:52.427099 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194eb77-a3b9-4579-8a50-368e076097e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:52.427848 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194eb77-a3b9-4579-8a50-368e076097e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:52.428261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194eb77-a3b9-4579-8a50-368e076097e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:52.428657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194eb77-a3b9-4579-8a50-368e076097e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:52.429535 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194eb77-a3b9-4579-8a50-368e076097e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:52.431160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194eb77-a3b9-4579-8a50-368e076097e2 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:52.446328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 954/1898] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:52.448786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194eb77-a3b9-4579-8a50-368e076097e2 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:52.464242 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194eb77-a3b9-4579-8a50-368e076097e2 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:52.480923 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e05e2c-9080-40b0-ae86-762e5e7bdbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:52.481975 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e05e2c-9080-40b0-ae86-762e5e7bdbe9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:52.482270 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e05e2c-9080-40b0-ae86-762e5e7bdbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:52.482533 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e05e2c-9080-40b0-ae86-762e5e7bdbe9 admin admin] PATH_INFO: `/v3/projects/ff1ff526da91452e88c9e9b3ed449523/users/c31ed423af8542d4adc27a172453c23b/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:52.499831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e05e2c-9080-40b0-ae86-762e5e7bdbe9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff1ff526da91452e88c9e9b3ed449523, user_id=c31ed423af8542d4adc27a172453c23b, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:52.503233 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e05e2c-9080-40b0-ae86-762e5e7bdbe9 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:52.564881 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 945/1899] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:07:52 2022] POST /identity/v3/users => generated 290 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:52.566193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31ab1f-49d5-436b-ab80-05f85c08aae8 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:52.580372 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31ab1f-49d5-436b-ab80-05f85c08aae8 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:52.585099 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e232d45-27c2-432c-9ad3-cfc88b9fb9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:52.595938 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e232d45-27c2-432c-9ad3-cfc88b9fb9d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:52.595938 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e232d45-27c2-432c-9ad3-cfc88b9fb9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:52.595938 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e232d45-27c2-432c-9ad3-cfc88b9fb9d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:52.595938 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e232d45-27c2-432c-9ad3-cfc88b9fb9d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:52.595938 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e232d45-27c2-432c-9ad3-cfc88b9fb9d9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:52.598789 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 946/1900] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:52.602750 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e232d45-27c2-432c-9ad3-cfc88b9fb9d9 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:52.613238 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e232d45-27c2-432c-9ad3-cfc88b9fb9d9 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:52.617901 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d8f8a-c85f-483c-b9d1-726efa6efa24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:52.618555 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d8f8a-c85f-483c-b9d1-726efa6efa24 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:52.618838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d8f8a-c85f-483c-b9d1-726efa6efa24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:52.619115 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d8f8a-c85f-483c-b9d1-726efa6efa24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:52.619837 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d8f8a-c85f-483c-b9d1-726efa6efa24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:52.621488 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d8f8a-c85f-483c-b9d1-726efa6efa24 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:52.632010 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 947/1901] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:52.636008 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d8f8a-c85f-483c-b9d1-726efa6efa24 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:52.646563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 955/1902] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:52 2022] PUT /identity/v3/projects/ff1ff526da91452e88c9e9b3ed449523/users/c31ed423af8542d4adc27a172453c23b/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:52.647315 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d8f8a-c85f-483c-b9d1-726efa6efa24 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:52.650471 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e05e2c-9080-40b0-ae86-762e5e7bdbe9 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:52.744508 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6500ce-1f29-4006-af4a-8659b2af102e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:52.745705 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6500ce-1f29-4006-af4a-8659b2af102e admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:52.746138 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6500ce-1f29-4006-af4a-8659b2af102e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:52.746561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6500ce-1f29-4006-af4a-8659b2af102e admin admin] PATH_INFO: `/v3/projects/edb569a8b39a437ea8be940df098268c/users/eafea4329d6243459dd3a6add1e91a2e/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:52.765475 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6500ce-1f29-4006-af4a-8659b2af102e admin admin] RBAC: Authorizing `identity:create_grant(project_id=edb569a8b39a437ea8be940df098268c, user_id=eafea4329d6243459dd3a6add1e91a2e, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:52.768464 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6500ce-1f29-4006-af4a-8659b2af102e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:52.782544 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e05e2c-9080-40b0-ae86-762e5e7bdbe9 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:52.788722 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba085cf8-ece5-4e1a-bad1-25af2d12a26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:52.789533 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba085cf8-ece5-4e1a-bad1-25af2d12a26f admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:52.790870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba085cf8-ece5-4e1a-bad1-25af2d12a26f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:52.790870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba085cf8-ece5-4e1a-bad1-25af2d12a26f admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:52.791290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba085cf8-ece5-4e1a-bad1-25af2d12a26f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:52.793810 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba085cf8-ece5-4e1a-bad1-25af2d12a26f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:52.807956 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 956/1903] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:52.807956 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22b14e-4bd2-4c7d-9eaf-5d993674a989 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:52.807956 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22b14e-4bd2-4c7d-9eaf-5d993674a989 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:52.807956 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22b14e-4bd2-4c7d-9eaf-5d993674a989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:52.842628 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-6c22b14e-4bd2-4c7d-9eaf-5d993674a989 None None] MFA Rules not processed for user `c41d51884a3b47318c6d05fba5918fa7`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:52.843392 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6c22b14e-4bd2-4c7d-9eaf-5d993674a989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:52.869846 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 948/1904] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:52 2022] PUT /identity/v3/projects/edb569a8b39a437ea8be940df098268c/users/eafea4329d6243459dd3a6add1e91a2e/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:07:52.880383 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6500ce-1f29-4006-af4a-8659b2af102e admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:52.977380 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 957/1905] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:52.979413 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431300c9-db21-4aca-9a13-3cbb17deb622 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:52.979814 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431300c9-db21-4aca-9a13-3cbb17deb622 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:52.980597 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431300c9-db21-4aca-9a13-3cbb17deb622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:53.003948 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-431300c9-db21-4aca-9a13-3cbb17deb622 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:53.005919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-431300c9-db21-4aca-9a13-3cbb17deb622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:53.057995 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6500ce-1f29-4006-af4a-8659b2af102e admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:53.057995 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc76417f-8bdd-4d60-9e40-7ece003df5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:53.057995 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76417f-8bdd-4d60-9e40-7ece003df5a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:53.057995 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76417f-8bdd-4d60-9e40-7ece003df5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:53.058601 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76417f-8bdd-4d60-9e40-7ece003df5a8 admin admin] PATH_INFO: `/v3/projects/ff1ff526da91452e88c9e9b3ed449523/users/c31ed423af8542d4adc27a172453c23b/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:53.059869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc76417f-8bdd-4d60-9e40-7ece003df5a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff1ff526da91452e88c9e9b3ed449523, user_id=c31ed423af8542d4adc27a172453c23b, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:53.062755 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc76417f-8bdd-4d60-9e40-7ece003df5a8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:53.080342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 949/1906] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:52 2022] PUT /identity/v3/projects/ff1ff526da91452e88c9e9b3ed449523/users/c31ed423af8542d4adc27a172453c23b/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:53.080342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc76417f-8bdd-4d60-9e40-7ece003df5a8 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:53.087894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fc76417f-8bdd-4d60-9e40-7ece003df5a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:53.098888 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 958/1907] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:53.098888 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecddbfc5-20e3-4d91-8289-e1d1a4d5df68 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:53.098888 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecddbfc5-20e3-4d91-8289-e1d1a4d5df68 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:53.098888 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecddbfc5-20e3-4d91-8289-e1d1a4d5df68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:53.155900 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ecddbfc5-20e3-4d91-8289-e1d1a4d5df68 None None] MFA Rules not processed for user `c31ed423af8542d4adc27a172453c23b`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:53.157300 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ecddbfc5-20e3-4d91-8289-e1d1a4d5df68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:53.182695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc76417f-8bdd-4d60-9e40-7ece003df5a8 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:53.187475 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef36fc7-8bcb-43ae-90e0-0a0e9781130a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:53.188147 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef36fc7-8bcb-43ae-90e0-0a0e9781130a admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:53.188471 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef36fc7-8bcb-43ae-90e0-0a0e9781130a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:53.188747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef36fc7-8bcb-43ae-90e0-0a0e9781130a admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:53.189625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef36fc7-8bcb-43ae-90e0-0a0e9781130a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:53.191556 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef36fc7-8bcb-43ae-90e0-0a0e9781130a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:53.235948 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 959/1908] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:53 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:53.245228 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecddbfc5-20e3-4d91-8289-e1d1a4d5df68 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:53.246754 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ecddbfc5-20e3-4d91-8289-e1d1a4d5df68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:53.301502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 950/1909] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:07:53 2022] POST /identity/v3/projects => generated 373 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:53.302450 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef36fc7-8bcb-43ae-90e0-0a0e9781130a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:53.305586 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1ef36fc7-8bcb-43ae-90e0-0a0e9781130a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:53.331116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecddbfc5-20e3-4d91-8289-e1d1a4d5df68 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:53.373160 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef36fc7-8bcb-43ae-90e0-0a0e9781130a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:53.377195 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa1043-3e63-4d88-8c0a-980dd92409e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:53.377852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa1043-3e63-4d88-8c0a-980dd92409e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:53.378145 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa1043-3e63-4d88-8c0a-980dd92409e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:53.378411 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa1043-3e63-4d88-8c0a-980dd92409e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:53.392598 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d903f1-857a-4e34-ab2e-6b3ad92a226d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:53.393577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d903f1-857a-4e34-ab2e-6b3ad92a226d service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:53.394037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d903f1-857a-4e34-ab2e-6b3ad92a226d service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:53.394494 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d903f1-857a-4e34-ab2e-6b3ad92a226d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:53.443802 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fa1043-3e63-4d88-8c0a-980dd92409e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:53.445297 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fa1043-3e63-4d88-8c0a-980dd92409e0 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:53.458141 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 951/1910] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:53 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:53.459234 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa1043-3e63-4d88-8c0a-980dd92409e0 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:53.467060 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d903f1-857a-4e34-ab2e-6b3ad92a226d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:53.468861 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d903f1-857a-4e34-ab2e-6b3ad92a226d service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:53.475758 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa1043-3e63-4d88-8c0a-980dd92409e0 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:53.480964 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 960/1911] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:53.481669 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd691c0f-25bc-4ed6-9d1a-18f6b34e6313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:53.482369 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd691c0f-25bc-4ed6-9d1a-18f6b34e6313 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:53.482657 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd691c0f-25bc-4ed6-9d1a-18f6b34e6313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:53.482936 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd691c0f-25bc-4ed6-9d1a-18f6b34e6313 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:53.483660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd691c0f-25bc-4ed6-9d1a-18f6b34e6313 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:53.485695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd691c0f-25bc-4ed6-9d1a-18f6b34e6313 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:53.566109 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d903f1-857a-4e34-ab2e-6b3ad92a226d service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:53.567529 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-47d903f1-857a-4e34-ab2e-6b3ad92a226d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:53.580159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d903f1-857a-4e34-ab2e-6b3ad92a226d service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:53.583925 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057305b9-7ae1-4790-a1f7-10badd0c51d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:53.584607 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057305b9-7ae1-4790-a1f7-10badd0c51d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:53.584891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057305b9-7ae1-4790-a1f7-10badd0c51d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:53.587115 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057305b9-7ae1-4790-a1f7-10badd0c51d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:53.587115 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-057305b9-7ae1-4790-a1f7-10badd0c51d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:53.645934 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:53.650226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057305b9-7ae1-4790-a1f7-10badd0c51d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:53.651704 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057305b9-7ae1-4790-a1f7-10badd0c51d0 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:53.661104 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 961/1912] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 20:07:53.684061 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 952/1913] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:07:53 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:07:53.691346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057305b9-7ae1-4790-a1f7-10badd0c51d0 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:53.702942 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057305b9-7ae1-4790-a1f7-10badd0c51d0 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:53.712484 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a909eff-087f-4a7d-8202-2917031132a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:53.713909 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a909eff-087f-4a7d-8202-2917031132a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:53.714465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a909eff-087f-4a7d-8202-2917031132a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:53.714925 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a909eff-087f-4a7d-8202-2917031132a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:53.715719 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a909eff-087f-4a7d-8202-2917031132a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:53.717450 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a909eff-087f-4a7d-8202-2917031132a9 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:53.729481 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 962/1914] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:07:53.733793 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd691c0f-25bc-4ed6-9d1a-18f6b34e6313 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:53.757801 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd691c0f-25bc-4ed6-9d1a-18f6b34e6313 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:53.769515 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721aaba-ce64-472c-8a8e-4d5d18feda40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:53.770903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721aaba-ce64-472c-8a8e-4d5d18feda40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:53.771906 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721aaba-ce64-472c-8a8e-4d5d18feda40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:53.772545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721aaba-ce64-472c-8a8e-4d5d18feda40 admin admin] PATH_INFO: `/v3/projects/eff258d9a57b49c1abd0dd440f79fcd6/users/48b728a8c896459d830bd07be28d6f02/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:53.798023 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7721aaba-ce64-472c-8a8e-4d5d18feda40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eff258d9a57b49c1abd0dd440f79fcd6, user_id=48b728a8c896459d830bd07be28d6f02, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:53.805999 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7721aaba-ce64-472c-8a8e-4d5d18feda40 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:53.992146 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 953/1915] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:53 2022] PUT /identity/v3/projects/eff258d9a57b49c1abd0dd440f79fcd6/users/48b728a8c896459d830bd07be28d6f02/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:53.997035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a909eff-087f-4a7d-8202-2917031132a9 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:54.008683 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a909eff-087f-4a7d-8202-2917031132a9 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:54.071240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab4418-784b-4650-80e4-1cc7f76faeef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:54.072169 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab4418-784b-4650-80e4-1cc7f76faeef admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:54.072529 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab4418-784b-4650-80e4-1cc7f76faeef admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:54.072806 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab4418-784b-4650-80e4-1cc7f76faeef admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:54.073536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab4418-784b-4650-80e4-1cc7f76faeef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:54.075163 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab4418-784b-4650-80e4-1cc7f76faeef admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:54.087040 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 963/1916] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:07:54.091179 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721aaba-ce64-472c-8a8e-4d5d18feda40 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:54.102125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721aaba-ce64-472c-8a8e-4d5d18feda40 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:54.106958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f5ac2-37c1-4148-b498-5c41de719220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:54.107890 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f5ac2-37c1-4148-b498-5c41de719220 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:54.108274 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f5ac2-37c1-4148-b498-5c41de719220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:54.108656 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f5ac2-37c1-4148-b498-5c41de719220 admin admin] PATH_INFO: `/v3/projects/eff258d9a57b49c1abd0dd440f79fcd6/users/48b728a8c896459d830bd07be28d6f02/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:54.110686 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730f5ac2-37c1-4148-b498-5c41de719220 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eff258d9a57b49c1abd0dd440f79fcd6, user_id=48b728a8c896459d830bd07be28d6f02, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:54.114033 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730f5ac2-37c1-4148-b498-5c41de719220 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:54.125999 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 954/1917] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:07:54 2022] PUT /identity/v3/projects/eff258d9a57b49c1abd0dd440f79fcd6/users/48b728a8c896459d830bd07be28d6f02/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:07:54.710251 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=82,keystone:INSERT=4,keystone:DELETE=14 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:55.217992 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:55.904365 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7019a03-4187-4c1f-93e5-2a18053ecb3c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:55.905439 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7019a03-4187-4c1f-93e5-2a18053ecb3c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:55.905719 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7019a03-4187-4c1f-93e5-2a18053ecb3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:55.943747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a7019a03-4187-4c1f-93e5-2a18053ecb3c None None] MFA Rules not processed for user `48b728a8c896459d830bd07be28d6f02`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:07:55.945822 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a7019a03-4187-4c1f-93e5-2a18053ecb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:07:56.042404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 964/1918] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:07:55 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:07:56.053381 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f5ac2-37c1-4148-b498-5c41de719220 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:56.121490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f5ac2-37c1-4148-b498-5c41de719220 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:56.128056 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3eb3b0-31d5-4508-98e5-9a3081f1d141 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:56.128830 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3eb3b0-31d5-4508-98e5-9a3081f1d141 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:56.129238 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3eb3b0-31d5-4508-98e5-9a3081f1d141 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:56.130112 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3eb3b0-31d5-4508-98e5-9a3081f1d141 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:56.199131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3eb3b0-31d5-4508-98e5-9a3081f1d141 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:56.200678 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3eb3b0-31d5-4508-98e5-9a3081f1d141 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:56.220633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 955/1919] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:07:57.017867 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:57.365666 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:58.806486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:59.158312 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:07:59.183666 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7019a03-4187-4c1f-93e5-2a18053ecb3c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:07:59.195408 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7019a03-4187-4c1f-93e5-2a18053ecb3c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:07:59.203495 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352947d-d005-4607-a5ea-3e296481086b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:07:59.205446 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352947d-d005-4607-a5ea-3e296481086b service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:07:59.205937 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352947d-d005-4607-a5ea-3e296481086b service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:07:59.206431 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352947d-d005-4607-a5ea-3e296481086b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:07:59.217545 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352947d-d005-4607-a5ea-3e296481086b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:07:59.219194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352947d-d005-4607-a5ea-3e296481086b service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:07:59.258402 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 965/1920] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:07:59 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:08:00.623889 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:01.219397 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3e769-3d0d-42ef-b1fa-a1a1610f17a7 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:01.220608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3e769-3d0d-42ef-b1fa-a1a1610f17a7 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:01.220608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3e769-3d0d-42ef-b1fa-a1a1610f17a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:01.249976 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b1e3e769-3d0d-42ef-b1fa-a1a1610f17a7 None None] MFA Rules not processed for user `eafea4329d6243459dd3a6add1e91a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:01.251320 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b1e3e769-3d0d-42ef-b1fa-a1a1610f17a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:01.332336 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 956/1921] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:08:01.337245 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352947d-d005-4607-a5ea-3e296481086b service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:01.452409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352947d-d005-4607-a5ea-3e296481086b service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:01.462835 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb51c5-e50f-43d8-99f5-c504e8b4fe38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:01.463520 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb51c5-e50f-43d8-99f5-c504e8b4fe38 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:01.463816 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb51c5-e50f-43d8-99f5-c504e8b4fe38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:01.464092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb51c5-e50f-43d8-99f5-c504e8b4fe38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:01.464835 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb51c5-e50f-43d8-99f5-c504e8b4fe38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:01.466782 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb51c5-e50f-43d8-99f5-c504e8b4fe38 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:01.686422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 966/1922] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:08:01 2022] POST /identity/v3/projects => generated 379 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:01.687402 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3e769-3d0d-42ef-b1fa-a1a1610f17a7 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:01.699824 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3e769-3d0d-42ef-b1fa-a1a1610f17a7 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:01.790928 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185d490-d83c-46c3-8b06-3fd62b62d4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:01.791642 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185d490-d83c-46c3-8b06-3fd62b62d4bf admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:01.791927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185d490-d83c-46c3-8b06-3fd62b62d4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:01.792202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185d490-d83c-46c3-8b06-3fd62b62d4bf admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:01.792936 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e185d490-d83c-46c3-8b06-3fd62b62d4bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:01.795417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e185d490-d83c-46c3-8b06-3fd62b62d4bf admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:01.922424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 957/1923] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:08:01 2022] POST /identity/v3/users => generated 296 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:08:01.929237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb51c5-e50f-43d8-99f5-c504e8b4fe38 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:01.944913 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb51c5-e50f-43d8-99f5-c504e8b4fe38 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:01.959357 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf01520d-81a0-4fb3-b71e-062a39f00a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:01.960688 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf01520d-81a0-4fb3-b71e-062a39f00a41 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:01.961131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf01520d-81a0-4fb3-b71e-062a39f00a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:01.961550 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf01520d-81a0-4fb3-b71e-062a39f00a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:01.967542 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf01520d-81a0-4fb3-b71e-062a39f00a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:01.969527 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf01520d-81a0-4fb3-b71e-062a39f00a41 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:02.000938 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 967/1924] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:02.008329 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185d490-d83c-46c3-8b06-3fd62b62d4bf admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:02.040384 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185d490-d83c-46c3-8b06-3fd62b62d4bf admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:02.045989 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e8cf7-bc03-4197-b549-e73522d86f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:02.047407 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e8cf7-bc03-4197-b549-e73522d86f41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:02.047666 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e8cf7-bc03-4197-b549-e73522d86f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:02.048099 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e8cf7-bc03-4197-b549-e73522d86f41 admin admin] PATH_INFO: `/v3/projects/95ab1ee032d74c4fbc1c544a09ffea80/users/c04d712cc71545ffad09b88a1392c5c7/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:02.067722 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e8cf7-bc03-4197-b549-e73522d86f41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95ab1ee032d74c4fbc1c544a09ffea80, user_id=c04d712cc71545ffad09b88a1392c5c7, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:02.070865 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727e8cf7-bc03-4197-b549-e73522d86f41 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:02.165921 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:02.181151 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 958/1925] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:08:02 2022] PUT /identity/v3/projects/95ab1ee032d74c4fbc1c544a09ffea80/users/c04d712cc71545ffad09b88a1392c5c7/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:02.185050 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf01520d-81a0-4fb3-b71e-062a39f00a41 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:02.206712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf01520d-81a0-4fb3-b71e-062a39f00a41 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:02.342399 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bed7f-1421-41fd-852d-0e19d0fbb62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:02.345738 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bed7f-1421-41fd-852d-0e19d0fbb62e admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:02.346592 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bed7f-1421-41fd-852d-0e19d0fbb62e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:02.347687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bed7f-1421-41fd-852d-0e19d0fbb62e admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:02.348690 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bed7f-1421-41fd-852d-0e19d0fbb62e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:02.350589 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bed7f-1421-41fd-852d-0e19d0fbb62e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:02.380943 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 968/1926] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:02.385273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e8cf7-bc03-4197-b549-e73522d86f41 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:02.400606 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e8cf7-bc03-4197-b549-e73522d86f41 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:02.424334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae72ae-e1d8-4876-9017-dedac45334aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:02.425505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae72ae-e1d8-4876-9017-dedac45334aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:02.427488 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae72ae-e1d8-4876-9017-dedac45334aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:02.428460 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae72ae-e1d8-4876-9017-dedac45334aa admin admin] PATH_INFO: `/v3/projects/95ab1ee032d74c4fbc1c544a09ffea80/users/c04d712cc71545ffad09b88a1392c5c7/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:02.436083 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bae72ae-e1d8-4876-9017-dedac45334aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=95ab1ee032d74c4fbc1c544a09ffea80, user_id=c04d712cc71545ffad09b88a1392c5c7, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:02.448628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bae72ae-e1d8-4876-9017-dedac45334aa admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:02.473850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 959/1927] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:08:02 2022] PUT /identity/v3/projects/95ab1ee032d74c4fbc1c544a09ffea80/users/c04d712cc71545ffad09b88a1392c5c7/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:02.698420 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:03.328880 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bed7f-1421-41fd-852d-0e19d0fbb62e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:03.344717 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bed7f-1421-41fd-852d-0e19d0fbb62e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:03.505058 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d172c9-9d21-429f-aa3d-7c82d6812719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:03.506326 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d172c9-9d21-429f-aa3d-7c82d6812719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:03.506791 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d172c9-9d21-429f-aa3d-7c82d6812719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:03.507215 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d172c9-9d21-429f-aa3d-7c82d6812719 admin admin] PATH_INFO: `/v3/users/65ec664c077445ad83d93c54db88b6ce` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:03.509920 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d172c9-9d21-429f-aa3d-7c82d6812719 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65ec664c077445ad83d93c54db88b6ce)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:03.513610 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d172c9-9d21-429f-aa3d-7c82d6812719 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:03.830098 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-56d172c9-9d21-429f-aa3d-7c82d6812719 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65ec664c077445ad83d93c54db88b6ce'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:03.832370 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-56d172c9-9d21-429f-aa3d-7c82d6812719 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65ec664c077445ad83d93c54db88b6ce'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:03.915823 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-56d172c9-9d21-429f-aa3d-7c82d6812719 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65ec664c077445ad83d93c54db88b6ce'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:03.916479 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-56d172c9-9d21-429f-aa3d-7c82d6812719 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65ec664c077445ad83d93c54db88b6ce'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:03.941296 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-56d172c9-9d21-429f-aa3d-7c82d6812719 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65ec664c077445ad83d93c54db88b6ce'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:03.984434 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367816c-01c7-4f2f-81cd-62c67b78f122 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:03.985307 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367816c-01c7-4f2f-81cd-62c67b78f122 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:03.986791 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 969/1928] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:08:03 2022] DELETE /identity/v3/users/65ec664c077445ad83d93c54db88b6ce => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:03.987605 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367816c-01c7-4f2f-81cd-62c67b78f122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:04.037791 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d367816c-01c7-4f2f-81cd-62c67b78f122 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:04.042416 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d367816c-01c7-4f2f-81cd-62c67b78f122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:04.184585 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 960/1929] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:08:04.212855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d172c9-9d21-429f-aa3d-7c82d6812719 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:04.214594 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-56d172c9-9d21-429f-aa3d-7c82d6812719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:04.331431 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d172c9-9d21-429f-aa3d-7c82d6812719 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:04.336013 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87e152-5ce5-4642-affa-51e30e3cd2f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:04.337280 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87e152-5ce5-4642-affa-51e30e3cd2f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:04.337719 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87e152-5ce5-4642-affa-51e30e3cd2f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:04.338127 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87e152-5ce5-4642-affa-51e30e3cd2f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:04.349875 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a87e152-5ce5-4642-affa-51e30e3cd2f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:04.352459 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a87e152-5ce5-4642-affa-51e30e3cd2f9 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:04.364137 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 970/1930] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:08:04.555330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:04.972495 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:05.476076 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367816c-01c7-4f2f-81cd-62c67b78f122 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:05.478989 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d367816c-01c7-4f2f-81cd-62c67b78f122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:05.505608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367816c-01c7-4f2f-81cd-62c67b78f122 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:05.505608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9873b7e2-f4e3-47f2-aab1-8b39dac74f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:05.505608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873b7e2-f4e3-47f2-aab1-8b39dac74f53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:05.505608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873b7e2-f4e3-47f2-aab1-8b39dac74f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:05.506058 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873b7e2-f4e3-47f2-aab1-8b39dac74f53 admin admin] PATH_INFO: `/v3/users/33465d9bb2424ae196ed04919b76ed46` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:05.507163 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9873b7e2-f4e3-47f2-aab1-8b39dac74f53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33465d9bb2424ae196ed04919b76ed46)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:05.510422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9873b7e2-f4e3-47f2-aab1-8b39dac74f53 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:05.816747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9873b7e2-f4e3-47f2-aab1-8b39dac74f53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33465d9bb2424ae196ed04919b76ed46'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:05.817760 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9873b7e2-f4e3-47f2-aab1-8b39dac74f53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33465d9bb2424ae196ed04919b76ed46'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:05.843387 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9873b7e2-f4e3-47f2-aab1-8b39dac74f53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33465d9bb2424ae196ed04919b76ed46'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:05.863883 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9873b7e2-f4e3-47f2-aab1-8b39dac74f53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33465d9bb2424ae196ed04919b76ed46'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:05.971725 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9873b7e2-f4e3-47f2-aab1-8b39dac74f53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33465d9bb2424ae196ed04919b76ed46'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:06.006078 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 961/1931] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:08:05 2022] DELETE /identity/v3/users/33465d9bb2424ae196ed04919b76ed46 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:06.012195 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0232d-fc8e-4cad-8edf-400d6e62da93 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:06.013933 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0232d-fc8e-4cad-8edf-400d6e62da93 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:06.014530 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0232d-fc8e-4cad-8edf-400d6e62da93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:06.074218 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d1a0232d-fc8e-4cad-8edf-400d6e62da93 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:06.077955 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d1a0232d-fc8e-4cad-8edf-400d6e62da93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:06.219720 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 971/1932] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:06.232087 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:06.240888 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9873b7e2-f4e3-47f2-aab1-8b39dac74f53 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:06.242417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9873b7e2-f4e3-47f2-aab1-8b39dac74f53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:06.394122 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9873b7e2-f4e3-47f2-aab1-8b39dac74f53 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:06.404656 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b762863-dfff-417a-b99d-e3e69269ffc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:06.405340 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b762863-dfff-417a-b99d-e3e69269ffc1 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:06.405621 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b762863-dfff-417a-b99d-e3e69269ffc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:06.405894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b762863-dfff-417a-b99d-e3e69269ffc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:06.443318 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b762863-dfff-417a-b99d-e3e69269ffc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:06.452408 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b762863-dfff-417a-b99d-e3e69269ffc1 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:06.471631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 962/1933] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:08:06.811222 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0232d-fc8e-4cad-8edf-400d6e62da93 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:06.812710 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d1a0232d-fc8e-4cad-8edf-400d6e62da93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:06.843010 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0232d-fc8e-4cad-8edf-400d6e62da93 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:06.864095 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019e530-073e-4d02-9f75-5045e7972780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:06.865339 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019e530-073e-4d02-9f75-5045e7972780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:06.865803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019e530-073e-4d02-9f75-5045e7972780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:06.866298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019e530-073e-4d02-9f75-5045e7972780 admin admin] PATH_INFO: `/v3/projects/2658c0810091420b9898887ba4cf7288` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:06.867921 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019e530-073e-4d02-9f75-5045e7972780 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2658c0810091420b9898887ba4cf7288)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:06.870221 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e019e530-073e-4d02-9f75-5045e7972780 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:07.051283 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e019e530-073e-4d02-9f75-5045e7972780 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2658c0810091420b9898887ba4cf7288', 'request_id': 'req-e019e530-073e-4d02-9f75-5045e7972780'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:07.126707 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e019e530-073e-4d02-9f75-5045e7972780 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2658c0810091420b9898887ba4cf7288', 'request_id': 'req-e019e530-073e-4d02-9f75-5045e7972780'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:07.132834 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e019e530-073e-4d02-9f75-5045e7972780 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2658c0810091420b9898887ba4cf7288', 'request_id': 'req-e019e530-073e-4d02-9f75-5045e7972780'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:07.137294 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b762863-dfff-417a-b99d-e3e69269ffc1 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:07.138754 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8b762863-dfff-417a-b99d-e3e69269ffc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:07.153715 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b762863-dfff-417a-b99d-e3e69269ffc1 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:07.173904 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e019e530-073e-4d02-9f75-5045e7972780 admin admin] The token cache is being invalidate because project 2658c0810091420b9898887ba4cf7288 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:08:07.174435 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e019e530-073e-4d02-9f75-5045e7972780 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:07.175299 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e019e530-073e-4d02-9f75-5045e7972780 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:07.180584 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 972/1934] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:08:06 2022] DELETE /identity/v3/projects/2658c0810091420b9898887ba4cf7288 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:07.181597 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019e530-073e-4d02-9f75-5045e7972780 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:07.183962 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e019e530-073e-4d02-9f75-5045e7972780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:07.236873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6bfa5-0b1b-464a-8620-34f2360db12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:07.238154 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6bfa5-0b1b-464a-8620-34f2360db12f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:07.238634 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6bfa5-0b1b-464a-8620-34f2360db12f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:07.239183 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6bfa5-0b1b-464a-8620-34f2360db12f admin admin] PATH_INFO: `/v3/projects/ebbc37447b7a44a88daf3f4da9466a64` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:07.240709 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e6bfa5-0b1b-464a-8620-34f2360db12f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebbc37447b7a44a88daf3f4da9466a64)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:07.242831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e6bfa5-0b1b-464a-8620-34f2360db12f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:07.261184 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019e530-073e-4d02-9f75-5045e7972780 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:07.277706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec1fe6-095c-4c1c-b6ff-79944137c125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:07.295767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec1fe6-095c-4c1c-b6ff-79944137c125 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:07.308043 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec1fe6-095c-4c1c-b6ff-79944137c125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:07.308043 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ec1fe6-095c-4c1c-b6ff-79944137c125 admin admin] PATH_INFO: `/v3/users/c31ed423af8542d4adc27a172453c23b` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:07.308043 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec1fe6-095c-4c1c-b6ff-79944137c125 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c31ed423af8542d4adc27a172453c23b)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:07.308043 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ec1fe6-095c-4c1c-b6ff-79944137c125 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:07.418111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-24e6bfa5-0b1b-464a-8620-34f2360db12f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebbc37447b7a44a88daf3f4da9466a64', 'request_id': 'req-24e6bfa5-0b1b-464a-8620-34f2360db12f'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:07.426693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-24e6bfa5-0b1b-464a-8620-34f2360db12f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebbc37447b7a44a88daf3f4da9466a64', 'request_id': 'req-24e6bfa5-0b1b-464a-8620-34f2360db12f'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:07.463424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-24e6bfa5-0b1b-464a-8620-34f2360db12f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebbc37447b7a44a88daf3f4da9466a64', 'request_id': 'req-24e6bfa5-0b1b-464a-8620-34f2360db12f'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:07.496814 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-37ec1fe6-095c-4c1c-b6ff-79944137c125 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c31ed423af8542d4adc27a172453c23b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:07.497541 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-37ec1fe6-095c-4c1c-b6ff-79944137c125 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c31ed423af8542d4adc27a172453c23b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:07.548833 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-24e6bfa5-0b1b-464a-8620-34f2360db12f admin admin] The token cache is being invalidate because project ebbc37447b7a44a88daf3f4da9466a64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:08:07.549221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-24e6bfa5-0b1b-464a-8620-34f2360db12f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:07.549917 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-24e6bfa5-0b1b-464a-8620-34f2360db12f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:07.551523 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 963/1935] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:08:07 2022] DELETE /identity/v3/projects/ebbc37447b7a44a88daf3f4da9466a64 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:07.556064 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6bfa5-0b1b-464a-8620-34f2360db12f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:07.557548 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-24e6bfa5-0b1b-464a-8620-34f2360db12f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:07.567195 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:07.632349 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6bfa5-0b1b-464a-8620-34f2360db12f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:07.635019 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-37ec1fe6-095c-4c1c-b6ff-79944137c125 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c31ed423af8542d4adc27a172453c23b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:07.635893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-37ec1fe6-095c-4c1c-b6ff-79944137c125 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c31ed423af8542d4adc27a172453c23b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:07.638001 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319bce05-8726-4292-a8d4-7ec4d946e5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:07.639091 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bce05-8726-4292-a8d4-7ec4d946e5a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:07.639540 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bce05-8726-4292-a8d4-7ec4d946e5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:07.639974 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319bce05-8726-4292-a8d4-7ec4d946e5a1 admin admin] PATH_INFO: `/v3/users/2ed0f50aa4604d74ae75571524acfeb4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:07.641412 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319bce05-8726-4292-a8d4-7ec4d946e5a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ed0f50aa4604d74ae75571524acfeb4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:07.644734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319bce05-8726-4292-a8d4-7ec4d946e5a1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:07.656172 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-37ec1fe6-095c-4c1c-b6ff-79944137c125 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c31ed423af8542d4adc27a172453c23b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:07.673546 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 973/1936] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:08:07 2022] DELETE /identity/v3/users/c31ed423af8542d4adc27a172453c23b => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:07.869456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-319bce05-8726-4292-a8d4-7ec4d946e5a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ed0f50aa4604d74ae75571524acfeb4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:07.870393 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-319bce05-8726-4292-a8d4-7ec4d946e5a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ed0f50aa4604d74ae75571524acfeb4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:07.892692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-319bce05-8726-4292-a8d4-7ec4d946e5a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ed0f50aa4604d74ae75571524acfeb4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:07.906056 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-319bce05-8726-4292-a8d4-7ec4d946e5a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ed0f50aa4604d74ae75571524acfeb4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:07.977323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-319bce05-8726-4292-a8d4-7ec4d946e5a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ed0f50aa4604d74ae75571524acfeb4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:07.988117 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 964/1937] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:08:07 2022] DELETE /identity/v3/users/2ed0f50aa4604d74ae75571524acfeb4 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:08:08.241792 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec1fe6-095c-4c1c-b6ff-79944137c125 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:08.246249 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-37ec1fe6-095c-4c1c-b6ff-79944137c125 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:08.347425 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ec1fe6-095c-4c1c-b6ff-79944137c125 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:08.418553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ede091-5fe1-4d5b-b2c2-5cf3efa8c4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:08.419470 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ede091-5fe1-4d5b-b2c2-5cf3efa8c4c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:08.419869 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ede091-5fe1-4d5b-b2c2-5cf3efa8c4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:08.420149 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ede091-5fe1-4d5b-b2c2-5cf3efa8c4c4 admin admin] PATH_INFO: `/v3/projects/ff1ff526da91452e88c9e9b3ed449523` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:08.421711 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ede091-5fe1-4d5b-b2c2-5cf3efa8c4c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff1ff526da91452e88c9e9b3ed449523)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:08.424193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ede091-5fe1-4d5b-b2c2-5cf3efa8c4c4 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:08.624167 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d4ede091-5fe1-4d5b-b2c2-5cf3efa8c4c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff1ff526da91452e88c9e9b3ed449523', 'request_id': 'req-d4ede091-5fe1-4d5b-b2c2-5cf3efa8c4c4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:08.737974 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d4ede091-5fe1-4d5b-b2c2-5cf3efa8c4c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff1ff526da91452e88c9e9b3ed449523', 'request_id': 'req-d4ede091-5fe1-4d5b-b2c2-5cf3efa8c4c4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:08.745691 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d4ede091-5fe1-4d5b-b2c2-5cf3efa8c4c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff1ff526da91452e88c9e9b3ed449523', 'request_id': 'req-d4ede091-5fe1-4d5b-b2c2-5cf3efa8c4c4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:08.763302 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d4ede091-5fe1-4d5b-b2c2-5cf3efa8c4c4 admin admin] The token cache is being invalidate because project ff1ff526da91452e88c9e9b3ed449523 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:08:08.763676 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d4ede091-5fe1-4d5b-b2c2-5cf3efa8c4c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:08.764659 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d4ede091-5fe1-4d5b-b2c2-5cf3efa8c4c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:08.771406 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 974/1938] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:08:08 2022] DELETE /identity/v3/projects/ff1ff526da91452e88c9e9b3ed449523 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:08.772783 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee5926-7786-4a27-bae4-8e2e42777e60 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:08.775916 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee5926-7786-4a27-bae4-8e2e42777e60 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:08.776423 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee5926-7786-4a27-bae4-8e2e42777e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:08.798551 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-28ee5926-7786-4a27-bae4-8e2e42777e60 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:08.800153 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-28ee5926-7786-4a27-bae4-8e2e42777e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:08.886001 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 965/1939] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:08.895818 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dcff2d-45a1-4011-8399-74fac7ea4175 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:08.896196 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dcff2d-45a1-4011-8399-74fac7ea4175 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:08.896619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dcff2d-45a1-4011-8399-74fac7ea4175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:08.919140 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-92dcff2d-45a1-4011-8399-74fac7ea4175 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:08.920445 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-92dcff2d-45a1-4011-8399-74fac7ea4175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:08.930375 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 975/1940] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:08.934256 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee5926-7786-4a27-bae4-8e2e42777e60 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:08.946915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee5926-7786-4a27-bae4-8e2e42777e60 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:08.952251 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a5633-568d-419a-933d-31a9f1827129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:08.953155 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a5633-568d-419a-933d-31a9f1827129 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:08.953621 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a5633-568d-419a-933d-31a9f1827129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:08.954306 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a5633-568d-419a-933d-31a9f1827129 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:08.955048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a5633-568d-419a-933d-31a9f1827129 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:08:08.955354 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a5633-568d-419a-933d-31a9f1827129 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:08.957032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a5633-568d-419a-933d-31a9f1827129 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:08.978714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 966/1941] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:08:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:08:08.980576 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dcff2d-45a1-4011-8399-74fac7ea4175 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:08.993744 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dcff2d-45a1-4011-8399-74fac7ea4175 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:09.000669 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec72f8-eecb-41c3-936e-565caed819d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:09.001924 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec72f8-eecb-41c3-936e-565caed819d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:09.002194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec72f8-eecb-41c3-936e-565caed819d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:09.002463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec72f8-eecb-41c3-936e-565caed819d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:09.003190 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec72f8-eecb-41c3-936e-565caed819d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:09.005155 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec72f8-eecb-41c3-936e-565caed819d3 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:09.140543 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 976/1942] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:08:08 2022] POST /identity/v3/projects => generated 407 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:09.142753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a5633-568d-419a-933d-31a9f1827129 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:09.153120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a5633-568d-419a-933d-31a9f1827129 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:09.210301 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:09.225776 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc1306-0071-4af3-92a2-d607761335c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:09.226715 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc1306-0071-4af3-92a2-d607761335c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:09.227144 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc1306-0071-4af3-92a2-d607761335c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:09.227578 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc1306-0071-4af3-92a2-d607761335c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:09.228590 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dc1306-0071-4af3-92a2-d607761335c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:09.230955 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dc1306-0071-4af3-92a2-d607761335c1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:09.239711 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec72f8-eecb-41c3-936e-565caed819d3 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:09.242510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-18ec72f8-eecb-41c3-936e-565caed819d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:09.265352 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec72f8-eecb-41c3-936e-565caed819d3 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:09.274288 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8862dd9-132f-4d57-b1a9-428d5f26ef4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:09.275185 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8862dd9-132f-4d57-b1a9-428d5f26ef4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:09.275470 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8862dd9-132f-4d57-b1a9-428d5f26ef4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:09.275744 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8862dd9-132f-4d57-b1a9-428d5f26ef4a admin admin] PATH_INFO: `/v3/projects/dbb3b7f14b6647e1b250dba5d6aead39` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:09.277158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8862dd9-132f-4d57-b1a9-428d5f26ef4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbb3b7f14b6647e1b250dba5d6aead39)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:09.279391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8862dd9-132f-4d57-b1a9-428d5f26ef4a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:09.318846 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 967/1943] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:08:09 2022] POST /identity/v3/users => generated 304 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:09.320115 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc1306-0071-4af3-92a2-d607761335c1 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:09.335027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc1306-0071-4af3-92a2-d607761335c1 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:09.340970 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef928fbc-c24a-46e7-bba0-73a05491c4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:09.341766 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef928fbc-c24a-46e7-bba0-73a05491c4fa admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:09.346794 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef928fbc-c24a-46e7-bba0-73a05491c4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:09.346794 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef928fbc-c24a-46e7-bba0-73a05491c4fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:09.346794 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef928fbc-c24a-46e7-bba0-73a05491c4fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:09.346794 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef928fbc-c24a-46e7-bba0-73a05491c4fa admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:09.358445 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 968/1944] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:09.363088 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef928fbc-c24a-46e7-bba0-73a05491c4fa admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:09.373970 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef928fbc-c24a-46e7-bba0-73a05491c4fa admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:09.378884 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e7044-668a-4b13-a1a8-95d2bceca6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:09.379556 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e7044-668a-4b13-a1a8-95d2bceca6e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:09.379838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e7044-668a-4b13-a1a8-95d2bceca6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:09.380112 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e7044-668a-4b13-a1a8-95d2bceca6e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:09.381057 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e7044-668a-4b13-a1a8-95d2bceca6e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:09.382521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e7044-668a-4b13-a1a8-95d2bceca6e0 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:09.396426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 969/1945] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:08:09.398154 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e7044-668a-4b13-a1a8-95d2bceca6e0 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:09.410666 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e7044-668a-4b13-a1a8-95d2bceca6e0 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:09.415693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49a5fc-d77b-47b4-92fb-09c5acc12eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:09.420465 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49a5fc-d77b-47b4-92fb-09c5acc12eee admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:09.420721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49a5fc-d77b-47b4-92fb-09c5acc12eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:09.421207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49a5fc-d77b-47b4-92fb-09c5acc12eee admin admin] PATH_INFO: `/v3/projects/c549ea77816d4fa88079e9087caef36e/users/998bbc74a41b47e6a561ce97c4d281b6/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:09.439109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b49a5fc-d77b-47b4-92fb-09c5acc12eee admin admin] RBAC: Authorizing `identity:create_grant(project_id=c549ea77816d4fa88079e9087caef36e, user_id=998bbc74a41b47e6a561ce97c4d281b6, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:09.441886 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b49a5fc-d77b-47b4-92fb-09c5acc12eee admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:09.443892 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b8862dd9-132f-4d57-b1a9-428d5f26ef4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbb3b7f14b6647e1b250dba5d6aead39', 'request_id': 'req-b8862dd9-132f-4d57-b1a9-428d5f26ef4a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:09.531460 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b8862dd9-132f-4d57-b1a9-428d5f26ef4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbb3b7f14b6647e1b250dba5d6aead39', 'request_id': 'req-b8862dd9-132f-4d57-b1a9-428d5f26ef4a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:09.541453 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 970/1946] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:08:09 2022] PUT /identity/v3/projects/c549ea77816d4fa88079e9087caef36e/users/998bbc74a41b47e6a561ce97c4d281b6/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:08:09.553454 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b8862dd9-132f-4d57-b1a9-428d5f26ef4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbb3b7f14b6647e1b250dba5d6aead39', 'request_id': 'req-b8862dd9-132f-4d57-b1a9-428d5f26ef4a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:09.555041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6a7cf-739d-4ae2-817d-cccdc0edbfea None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:09.557089 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6a7cf-739d-4ae2-817d-cccdc0edbfea None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:09.557693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6a7cf-739d-4ae2-817d-cccdc0edbfea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:09.574653 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b8862dd9-132f-4d57-b1a9-428d5f26ef4a admin admin] The token cache is being invalidate because project dbb3b7f14b6647e1b250dba5d6aead39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:08:09.574997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b8862dd9-132f-4d57-b1a9-428d5f26ef4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:09.575894 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b8862dd9-132f-4d57-b1a9-428d5f26ef4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:09.577530 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 977/1947] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:08:09 2022] DELETE /identity/v3/projects/dbb3b7f14b6647e1b250dba5d6aead39 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:09.584516 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524ab63-139a-4e23-9371-8a14a4239d2d None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:09.584516 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524ab63-139a-4e23-9371-8a14a4239d2d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:09.584516 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524ab63-139a-4e23-9371-8a14a4239d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:09.608973 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c524ab63-139a-4e23-9371-8a14a4239d2d None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:09.612269 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c524ab63-139a-4e23-9371-8a14a4239d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:09.617395 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-79f6a7cf-739d-4ae2-817d-cccdc0edbfea None None] MFA Rules not processed for user `998bbc74a41b47e6a561ce97c4d281b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:09.618616 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-79f6a7cf-739d-4ae2-817d-cccdc0edbfea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:09.705320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 978/1948] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:09.714846 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae077c9a-56e8-4624-8dd5-c70964371732 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:09.715216 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae077c9a-56e8-4624-8dd5-c70964371732 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:09.715629 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae077c9a-56e8-4624-8dd5-c70964371732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:09.744785 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 971/1949] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:09 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:08:09.746519 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6a7cf-739d-4ae2-817d-cccdc0edbfea None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:09.750504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-79f6a7cf-739d-4ae2-817d-cccdc0edbfea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:09.757853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ae077c9a-56e8-4624-8dd5-c70964371732 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:09.761424 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ae077c9a-56e8-4624-8dd5-c70964371732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:09.777746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6a7cf-739d-4ae2-817d-cccdc0edbfea None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:09.788125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 979/1950] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:09.788425 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae077c9a-56e8-4624-8dd5-c70964371732 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:09.797060 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708564ee-5d40-4811-8a20-187b7b0d8bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:09.798148 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708564ee-5d40-4811-8a20-187b7b0d8bfd admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:09.798435 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708564ee-5d40-4811-8a20-187b7b0d8bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:09.798738 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708564ee-5d40-4811-8a20-187b7b0d8bfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:09.799443 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708564ee-5d40-4811-8a20-187b7b0d8bfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:09.801560 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708564ee-5d40-4811-8a20-187b7b0d8bfd admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:09.808009 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae077c9a-56e8-4624-8dd5-c70964371732 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:09.815242 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5fb89-9646-43f5-a71d-fc974642024d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:09.815978 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5fb89-9646-43f5-a71d-fc974642024d admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:09.816968 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5fb89-9646-43f5-a71d-fc974642024d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:09.817262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa5fb89-9646-43f5-a71d-fc974642024d admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:09.818258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa5fb89-9646-43f5-a71d-fc974642024d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:08:09.818620 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa5fb89-9646-43f5-a71d-fc974642024d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:09.822615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa5fb89-9646-43f5-a71d-fc974642024d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:09.848254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 980/1951] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 20:08:09.851045 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5fb89-9646-43f5-a71d-fc974642024d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:09.865599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa5fb89-9646-43f5-a71d-fc974642024d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:09.872501 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c110562-bd62-4c9a-a0a5-acfd979f17bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:09.873173 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c110562-bd62-4c9a-a0a5-acfd979f17bc admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:09.873953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c110562-bd62-4c9a-a0a5-acfd979f17bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:09.873953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c110562-bd62-4c9a-a0a5-acfd979f17bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:09.874465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c110562-bd62-4c9a-a0a5-acfd979f17bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:09.876668 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c110562-bd62-4c9a-a0a5-acfd979f17bc admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:10.032136 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 972/1952] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:08:09 2022] POST /identity/v3/projects => generated 407 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:10.041948 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 981/1953] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:08:09 2022] POST /identity/v3/projects => generated 387 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:10.046246 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708564ee-5d40-4811-8a20-187b7b0d8bfd admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:10.052274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c110562-bd62-4c9a-a0a5-acfd979f17bc admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:10.173817 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708564ee-5d40-4811-8a20-187b7b0d8bfd admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:10.215277 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c110562-bd62-4c9a-a0a5-acfd979f17bc admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:10.216452 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29019229-88f4-487e-ba8e-4c8f20006a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:10.217672 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29019229-88f4-487e-ba8e-4c8f20006a26 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:10.218717 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29019229-88f4-487e-ba8e-4c8f20006a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:10.218717 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29019229-88f4-487e-ba8e-4c8f20006a26 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:10.220183 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29019229-88f4-487e-ba8e-4c8f20006a26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:10.222424 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29019229-88f4-487e-ba8e-4c8f20006a26 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:10.291565 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:10.292032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b3c65-f53c-4f3c-a6bb-e4f1917bfb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:10.292032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b3c65-f53c-4f3c-a6bb-e4f1917bfb96 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:10.292032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b3c65-f53c-4f3c-a6bb-e4f1917bfb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:10.292032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b3c65-f53c-4f3c-a6bb-e4f1917bfb96 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:10.292416 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121b3c65-f53c-4f3c-a6bb-e4f1917bfb96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:10.292416 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121b3c65-f53c-4f3c-a6bb-e4f1917bfb96 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:10.549999 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 982/1954] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:08:10 2022] POST /identity/v3/users => generated 294 bytes in 498 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:10.555287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 973/1955] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:08:10 2022] POST /identity/v3/users => generated 310 bytes in 510 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:10.560051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29019229-88f4-487e-ba8e-4c8f20006a26 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:10.563843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b3c65-f53c-4f3c-a6bb-e4f1917bfb96 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:10.567639 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29019229-88f4-487e-ba8e-4c8f20006a26 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:10.586694 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b3c65-f53c-4f3c-a6bb-e4f1917bfb96 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:10.589941 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa1563-b05d-4019-be29-82d21dd7632b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:10.591914 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa1563-b05d-4019-be29-82d21dd7632b admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:10.593438 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa1563-b05d-4019-be29-82d21dd7632b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:10.594157 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa1563-b05d-4019-be29-82d21dd7632b admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:10.596695 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa1563-b05d-4019-be29-82d21dd7632b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:10.602299 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41fc80f-6470-4a78-8ec4-1c655b890806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:10.602299 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fc80f-6470-4a78-8ec4-1c655b890806 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:10.602680 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa1563-b05d-4019-be29-82d21dd7632b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:10.602775 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fc80f-6470-4a78-8ec4-1c655b890806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:10.602775 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fc80f-6470-4a78-8ec4-1c655b890806 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:10.602775 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41fc80f-6470-4a78-8ec4-1c655b890806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:10.604436 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41fc80f-6470-4a78-8ec4-1c655b890806 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:10.623066 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 983/1956] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:10.624520 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa1563-b05d-4019-be29-82d21dd7632b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:10.635218 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 974/1957] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:10.636483 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41fc80f-6470-4a78-8ec4-1c655b890806 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:10.639323 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa1563-b05d-4019-be29-82d21dd7632b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:10.645382 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238a234-fa00-468b-b179-a6cee8419c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:10.647852 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238a234-fa00-468b-b179-a6cee8419c57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:10.648152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238a234-fa00-468b-b179-a6cee8419c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:10.648459 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238a234-fa00-468b-b179-a6cee8419c57 admin admin] PATH_INFO: `/v3/projects/eed10daaed984c74adcd681a372b4ce6/users/dc1da801397c48a2831d83bc2f0ece0c/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:10.653011 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41fc80f-6470-4a78-8ec4-1c655b890806 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:10.659643 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8adf8-d973-4751-8781-e6b5ab6c669c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:10.660768 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8adf8-d973-4751-8781-e6b5ab6c669c admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:10.661392 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8adf8-d973-4751-8781-e6b5ab6c669c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:10.661873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8adf8-d973-4751-8781-e6b5ab6c669c admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:10.689837 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8adf8-d973-4751-8781-e6b5ab6c669c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:10.689837 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8adf8-d973-4751-8781-e6b5ab6c669c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:10.690941 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238a234-fa00-468b-b179-a6cee8419c57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eed10daaed984c74adcd681a372b4ce6, user_id=dc1da801397c48a2831d83bc2f0ece0c, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:10.690941 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1238a234-fa00-468b-b179-a6cee8419c57 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:10.691420 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 975/1958] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:10.691420 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8adf8-d973-4751-8781-e6b5ab6c669c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:10.703012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8adf8-d973-4751-8781-e6b5ab6c669c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:10.709113 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a800e6-63fe-408a-a439-751674ee4875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:10.712920 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a800e6-63fe-408a-a439-751674ee4875 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:10.712920 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a800e6-63fe-408a-a439-751674ee4875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:10.712920 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a800e6-63fe-408a-a439-751674ee4875 admin admin] PATH_INFO: `/v3/projects/020e6b7e0123493b833638df28caea64/users/86f7586797774479842f4999de19b66a/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:10.729080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a800e6-63fe-408a-a439-751674ee4875 admin admin] RBAC: Authorizing `identity:create_grant(project_id=020e6b7e0123493b833638df28caea64, user_id=86f7586797774479842f4999de19b66a, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:10.732653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a800e6-63fe-408a-a439-751674ee4875 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:10.834748 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 984/1959] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:08:10 2022] PUT /identity/v3/projects/eed10daaed984c74adcd681a372b4ce6/users/dc1da801397c48a2831d83bc2f0ece0c/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:10.839107 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238a234-fa00-468b-b179-a6cee8419c57 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:10.867661 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 976/1960] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:08:10 2022] PUT /identity/v3/projects/020e6b7e0123493b833638df28caea64/users/86f7586797774479842f4999de19b66a/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:10.878057 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ec0cd-06db-4e5e-82c2-31e807b01453 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:10.878526 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ec0cd-06db-4e5e-82c2-31e807b01453 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:10.879068 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ec0cd-06db-4e5e-82c2-31e807b01453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:10.933619 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-508ec0cd-06db-4e5e-82c2-31e807b01453 None None] MFA Rules not processed for user `86f7586797774479842f4999de19b66a`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:10.936384 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-508ec0cd-06db-4e5e-82c2-31e807b01453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:10.953040 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1238a234-fa00-468b-b179-a6cee8419c57 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:11.100494 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3682dc2e-45b4-41f6-9b9f-2fa3af6f03c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:11.101290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682dc2e-45b4-41f6-9b9f-2fa3af6f03c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:11.101573 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682dc2e-45b4-41f6-9b9f-2fa3af6f03c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:11.102129 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682dc2e-45b4-41f6-9b9f-2fa3af6f03c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:11.102585 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3682dc2e-45b4-41f6-9b9f-2fa3af6f03c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:11.104255 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3682dc2e-45b4-41f6-9b9f-2fa3af6f03c9 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:11.110918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 977/1961] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:08:11.115555 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ec0cd-06db-4e5e-82c2-31e807b01453 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:11.121378 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 985/1962] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:08:11.127107 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3682dc2e-45b4-41f6-9b9f-2fa3af6f03c9 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:11.127667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ec0cd-06db-4e5e-82c2-31e807b01453 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:11.133044 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2669bc-b5d8-4214-a969-49cd0979ba79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:11.133839 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2669bc-b5d8-4214-a969-49cd0979ba79 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:11.134129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2669bc-b5d8-4214-a969-49cd0979ba79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:11.134576 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2669bc-b5d8-4214-a969-49cd0979ba79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:11.135475 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2669bc-b5d8-4214-a969-49cd0979ba79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:11.137676 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2669bc-b5d8-4214-a969-49cd0979ba79 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:11.140445 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3682dc2e-45b4-41f6-9b9f-2fa3af6f03c9 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:11.152165 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9d28e-6aed-4735-a056-dceacfe70bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:11.153087 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9d28e-6aed-4735-a056-dceacfe70bd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:11.153383 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9d28e-6aed-4735-a056-dceacfe70bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:11.153661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9d28e-6aed-4735-a056-dceacfe70bd7 admin admin] PATH_INFO: `/v3/projects/eed10daaed984c74adcd681a372b4ce6/users/dc1da801397c48a2831d83bc2f0ece0c/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:11.157918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9d28e-6aed-4735-a056-dceacfe70bd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eed10daaed984c74adcd681a372b4ce6, user_id=dc1da801397c48a2831d83bc2f0ece0c, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:11.163779 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9d28e-6aed-4735-a056-dceacfe70bd7 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:11.184462 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 986/1963] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:08:11 2022] PUT /identity/v3/projects/eed10daaed984c74adcd681a372b4ce6/users/dc1da801397c48a2831d83bc2f0ece0c/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:08:11.197111 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a772914-9e00-4bb3-9079-42121b70e008 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:11.197409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a772914-9e00-4bb3-9079-42121b70e008 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:11.197691 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a772914-9e00-4bb3-9079-42121b70e008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:11.230378 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-0a772914-9e00-4bb3-9079-42121b70e008 None None] MFA Rules not processed for user `dc1da801397c48a2831d83bc2f0ece0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:11.231676 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0a772914-9e00-4bb3-9079-42121b70e008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:11.287229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 978/1964] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:08:11 2022] POST /identity/v3/projects => generated 389 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:08:11.290905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2669bc-b5d8-4214-a969-49cd0979ba79 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:11.331598 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 987/1965] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:11 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:08:11.343898 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a772914-9e00-4bb3-9079-42121b70e008 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:11.346073 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0a772914-9e00-4bb3-9079-42121b70e008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:11.382792 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2669bc-b5d8-4214-a969-49cd0979ba79 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:11.395162 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350d58a-dd71-4cce-b048-45a2de42c33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:11.395890 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350d58a-dd71-4cce-b048-45a2de42c33b admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:11.396553 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350d58a-dd71-4cce-b048-45a2de42c33b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:11.397324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350d58a-dd71-4cce-b048-45a2de42c33b admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:11.399140 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350d58a-dd71-4cce-b048-45a2de42c33b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:11.401294 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350d58a-dd71-4cce-b048-45a2de42c33b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:11.423762 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a772914-9e00-4bb3-9079-42121b70e008 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:11.480253 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4d69d-c516-4c3b-92f3-eb72cf0e006d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:11.481115 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4d69d-c516-4c3b-92f3-eb72cf0e006d service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:11.481459 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4d69d-c516-4c3b-92f3-eb72cf0e006d service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:11.481857 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4d69d-c516-4c3b-92f3-eb72cf0e006d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:11.493355 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 979/1966] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:08:11 2022] POST /identity/v3/users => generated 301 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:11.497435 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350d58a-dd71-4cce-b048-45a2de42c33b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:11.514077 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350d58a-dd71-4cce-b048-45a2de42c33b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:11.521356 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b2c11-4223-4939-a1f4-e8a776da835c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:11.522098 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b2c11-4223-4939-a1f4-e8a776da835c admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:11.522371 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b2c11-4223-4939-a1f4-e8a776da835c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:11.522755 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b2c11-4223-4939-a1f4-e8a776da835c admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:11.526198 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b2c11-4223-4939-a1f4-e8a776da835c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:11.528224 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b2c11-4223-4939-a1f4-e8a776da835c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:11.549914 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 980/1967] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:11.550961 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b2c11-4223-4939-a1f4-e8a776da835c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:11.566336 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b2c11-4223-4939-a1f4-e8a776da835c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:11.572061 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcba8ba-2d84-4a53-91d0-33a18f5ed82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:11.573016 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcba8ba-2d84-4a53-91d0-33a18f5ed82c admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:11.573297 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcba8ba-2d84-4a53-91d0-33a18f5ed82c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:11.573573 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcba8ba-2d84-4a53-91d0-33a18f5ed82c admin admin] PATH_INFO: `/v3/projects/f61ec7306b9b457a9d1b4a5e32e2a2e4/users/baf607bf43c84c579b275a23507438a4/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:11.574821 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4d69d-c516-4c3b-92f3-eb72cf0e006d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:11.576778 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4d69d-c516-4c3b-92f3-eb72cf0e006d service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:11.588637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:11.595379 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcba8ba-2d84-4a53-91d0-33a18f5ed82c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f61ec7306b9b457a9d1b4a5e32e2a2e4, user_id=baf607bf43c84c579b275a23507438a4, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:11.598963 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcba8ba-2d84-4a53-91d0-33a18f5ed82c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:11.600248 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 988/1968] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:11 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:08:11.692644 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 981/1969] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:08:11 2022] PUT /identity/v3/projects/f61ec7306b9b457a9d1b4a5e32e2a2e4/users/baf607bf43c84c579b275a23507438a4/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:11.697112 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4d69d-c516-4c3b-92f3-eb72cf0e006d service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:11.708943 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4d69d-c516-4c3b-92f3-eb72cf0e006d service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:11.771673 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed781ba-1357-4912-9552-6b5ab3bed4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:11.772489 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed781ba-1357-4912-9552-6b5ab3bed4b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:11.773046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed781ba-1357-4912-9552-6b5ab3bed4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:11.773423 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed781ba-1357-4912-9552-6b5ab3bed4b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:11.775461 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed781ba-1357-4912-9552-6b5ab3bed4b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:11.777271 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed781ba-1357-4912-9552-6b5ab3bed4b4 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:11.791401 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 989/1970] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:11.793050 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcba8ba-2d84-4a53-91d0-33a18f5ed82c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:11.804647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcba8ba-2d84-4a53-91d0-33a18f5ed82c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:11.810875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b0d69-d93a-4bfa-bece-57e580c924f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:11.811849 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b0d69-d93a-4bfa-bece-57e580c924f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:11.812227 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b0d69-d93a-4bfa-bece-57e580c924f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:11.812592 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b0d69-d93a-4bfa-bece-57e580c924f5 admin admin] PATH_INFO: `/v3/projects/f61ec7306b9b457a9d1b4a5e32e2a2e4/users/baf607bf43c84c579b275a23507438a4/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:11.816733 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b0d69-d93a-4bfa-bece-57e580c924f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f61ec7306b9b457a9d1b4a5e32e2a2e4, user_id=baf607bf43c84c579b275a23507438a4, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:11.819659 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b0d69-d93a-4bfa-bece-57e580c924f5 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:11.834635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 982/1971] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:08:11 2022] PUT /identity/v3/projects/f61ec7306b9b457a9d1b4a5e32e2a2e4/users/baf607bf43c84c579b275a23507438a4/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:11.847153 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca07a9-f8df-45ef-9d11-b0884440f665 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:11.850696 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca07a9-f8df-45ef-9d11-b0884440f665 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:11.850696 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca07a9-f8df-45ef-9d11-b0884440f665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:11.890662 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-62ca07a9-f8df-45ef-9d11-b0884440f665 None None] MFA Rules not processed for user `baf607bf43c84c579b275a23507438a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:11.892598 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-62ca07a9-f8df-45ef-9d11-b0884440f665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:11.927093 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41253e-c4a7-4e20-9f46-48f7f3d709b4 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:11.927563 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41253e-c4a7-4e20-9f46-48f7f3d709b4 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:11.927686 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41253e-c4a7-4e20-9f46-48f7f3d709b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:11.971980 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-cf41253e-c4a7-4e20-9f46-48f7f3d709b4 None None] MFA Rules not processed for user `c04d712cc71545ffad09b88a1392c5c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:11.973308 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-cf41253e-c4a7-4e20-9f46-48f7f3d709b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:11.987062 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 990/1972] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:11 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:11.987062 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca07a9-f8df-45ef-9d11-b0884440f665 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:12.042264 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca07a9-f8df-45ef-9d11-b0884440f665 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:12.046070 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8d43e-4f55-4b81-91ba-0bc06420def4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:12.046735 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8d43e-4f55-4b81-91ba-0bc06420def4 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:12.047018 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8d43e-4f55-4b81-91ba-0bc06420def4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:12.047298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8d43e-4f55-4b81-91ba-0bc06420def4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:12.055105 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 983/1973] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:11 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:12.056017 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41253e-c4a7-4e20-9f46-48f7f3d709b4 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:12.058560 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-cf41253e-c4a7-4e20-9f46-48f7f3d709b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:12.071853 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41253e-c4a7-4e20-9f46-48f7f3d709b4 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:12.078190 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570bd9e-2ae1-4008-969b-a320cdaf2a41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:12.078985 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570bd9e-2ae1-4008-969b-a320cdaf2a41 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:12.079331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570bd9e-2ae1-4008-969b-a320cdaf2a41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:12.079722 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570bd9e-2ae1-4008-969b-a320cdaf2a41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:12.090898 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a570bd9e-2ae1-4008-969b-a320cdaf2a41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:12.092748 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a570bd9e-2ae1-4008-969b-a320cdaf2a41 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:12.106793 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 984/1974] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:12 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:08:12.107726 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570bd9e-2ae1-4008-969b-a320cdaf2a41 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:12.109168 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a570bd9e-2ae1-4008-969b-a320cdaf2a41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:12.115600 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8d43e-4f55-4b81-91ba-0bc06420def4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:12.120404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8d43e-4f55-4b81-91ba-0bc06420def4 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:12.144450 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 991/1975] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:11 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:08:12.239243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570bd9e-2ae1-4008-969b-a320cdaf2a41 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:12.249785 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14be2f-86ce-41ad-b633-f27abe66307c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:12.254510 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14be2f-86ce-41ad-b633-f27abe66307c service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:12.254510 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14be2f-86ce-41ad-b633-f27abe66307c service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:12.254510 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14be2f-86ce-41ad-b633-f27abe66307c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:12.257653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4d14be2f-86ce-41ad-b633-f27abe66307c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:12.349322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14be2f-86ce-41ad-b633-f27abe66307c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:12.351014 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14be2f-86ce-41ad-b633-f27abe66307c service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:12.361656 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 985/1976] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:08:12.445781 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:12.935659 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8d43e-4f55-4b81-91ba-0bc06420def4 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:13.026769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8d43e-4f55-4b81-91ba-0bc06420def4 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:13.032833 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e797a8-d129-4ca7-b900-09ddaec7aed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:13.033768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e797a8-d129-4ca7-b900-09ddaec7aed4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:13.034052 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e797a8-d129-4ca7-b900-09ddaec7aed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:13.034491 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e797a8-d129-4ca7-b900-09ddaec7aed4 admin admin] PATH_INFO: `/v3/users/86f7586797774479842f4999de19b66a` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:13.035670 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e797a8-d129-4ca7-b900-09ddaec7aed4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86f7586797774479842f4999de19b66a)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:13.037651 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e797a8-d129-4ca7-b900-09ddaec7aed4 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:13.301731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-44e797a8-d129-4ca7-b900-09ddaec7aed4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86f7586797774479842f4999de19b66a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:13.302482 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-44e797a8-d129-4ca7-b900-09ddaec7aed4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86f7586797774479842f4999de19b66a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:13.405974 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-44e797a8-d129-4ca7-b900-09ddaec7aed4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86f7586797774479842f4999de19b66a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:13.406625 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-44e797a8-d129-4ca7-b900-09ddaec7aed4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86f7586797774479842f4999de19b66a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:13.420088 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-44e797a8-d129-4ca7-b900-09ddaec7aed4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86f7586797774479842f4999de19b66a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:13.433620 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 992/1977] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:08:12 2022] DELETE /identity/v3/users/86f7586797774479842f4999de19b66a => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:13.439013 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbd6a1-ceb5-4baf-ac37-4c49b2f8cb97 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:13.439459 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbd6a1-ceb5-4baf-ac37-4c49b2f8cb97 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:13.442136 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbd6a1-ceb5-4baf-ac37-4c49b2f8cb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:13.458622 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d7bbd6a1-ceb5-4baf-ac37-4c49b2f8cb97 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:13.460915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d7bbd6a1-ceb5-4baf-ac37-4c49b2f8cb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:13.530637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 986/1978] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:13.543396 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e797a8-d129-4ca7-b900-09ddaec7aed4 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:13.545021 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-44e797a8-d129-4ca7-b900-09ddaec7aed4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:13.609511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e797a8-d129-4ca7-b900-09ddaec7aed4 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:13.613831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575114f-09dc-42f9-8eb3-539d4f7ccc98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:13.614599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575114f-09dc-42f9-8eb3-539d4f7ccc98 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:13.614942 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575114f-09dc-42f9-8eb3-539d4f7ccc98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:13.615282 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575114f-09dc-42f9-8eb3-539d4f7ccc98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:13.617842 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:13.628037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e575114f-09dc-42f9-8eb3-539d4f7ccc98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:13.629591 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e575114f-09dc-42f9-8eb3-539d4f7ccc98 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:13.642504 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 993/1979] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:08:14.168855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:14.241732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbd6a1-ceb5-4baf-ac37-4c49b2f8cb97 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:14.243611 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d7bbd6a1-ceb5-4baf-ac37-4c49b2f8cb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:14.257596 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbd6a1-ceb5-4baf-ac37-4c49b2f8cb97 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:14.262468 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276c8ac-9421-4f15-b56f-3d72dc6c2583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:14.263336 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276c8ac-9421-4f15-b56f-3d72dc6c2583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:14.263769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276c8ac-9421-4f15-b56f-3d72dc6c2583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:14.264277 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1276c8ac-9421-4f15-b56f-3d72dc6c2583 admin admin] PATH_INFO: `/v3/projects/020e6b7e0123493b833638df28caea64` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:14.265713 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1276c8ac-9421-4f15-b56f-3d72dc6c2583 admin admin] RBAC: Authorizing `identity:delete_project(project_id=020e6b7e0123493b833638df28caea64)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:14.268418 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1276c8ac-9421-4f15-b56f-3d72dc6c2583 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:14.343584 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:14.363330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-1276c8ac-9421-4f15-b56f-3d72dc6c2583 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '020e6b7e0123493b833638df28caea64', 'request_id': 'req-1276c8ac-9421-4f15-b56f-3d72dc6c2583'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:14.370137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-1276c8ac-9421-4f15-b56f-3d72dc6c2583 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '020e6b7e0123493b833638df28caea64', 'request_id': 'req-1276c8ac-9421-4f15-b56f-3d72dc6c2583'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:14.375926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-1276c8ac-9421-4f15-b56f-3d72dc6c2583 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '020e6b7e0123493b833638df28caea64', 'request_id': 'req-1276c8ac-9421-4f15-b56f-3d72dc6c2583'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:14.439644 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-1276c8ac-9421-4f15-b56f-3d72dc6c2583 admin admin] The token cache is being invalidate because project 020e6b7e0123493b833638df28caea64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:08:14.440350 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-1276c8ac-9421-4f15-b56f-3d72dc6c2583 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:14.440632 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-1276c8ac-9421-4f15-b56f-3d72dc6c2583 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:14.442508 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 987/1980] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:08:14 2022] DELETE /identity/v3/projects/020e6b7e0123493b833638df28caea64 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:14.447972 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575114f-09dc-42f9-8eb3-539d4f7ccc98 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:14.477340 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575114f-09dc-42f9-8eb3-539d4f7ccc98 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:14.543076 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:14.568573 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d0f75-a422-41bc-aa1d-973c7b27e394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:14.571350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d0f75-a422-41bc-aa1d-973c7b27e394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:14.571350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d0f75-a422-41bc-aa1d-973c7b27e394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:14.571350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d0f75-a422-41bc-aa1d-973c7b27e394 admin admin] PATH_INFO: `/v3/users/baf607bf43c84c579b275a23507438a4` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:14.571350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d0f75-a422-41bc-aa1d-973c7b27e394 admin admin] RBAC: Authorizing `identity:delete_user(user_id=baf607bf43c84c579b275a23507438a4)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:14.573565 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d0f75-a422-41bc-aa1d-973c7b27e394 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:14.880121 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-169d0f75-a422-41bc-aa1d-973c7b27e394 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baf607bf43c84c579b275a23507438a4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:14.881188 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-169d0f75-a422-41bc-aa1d-973c7b27e394 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baf607bf43c84c579b275a23507438a4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:14.994394 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-169d0f75-a422-41bc-aa1d-973c7b27e394 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baf607bf43c84c579b275a23507438a4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:14.995691 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-169d0f75-a422-41bc-aa1d-973c7b27e394 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baf607bf43c84c579b275a23507438a4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:15.014296 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-169d0f75-a422-41bc-aa1d-973c7b27e394 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baf607bf43c84c579b275a23507438a4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:15.029935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 994/1981] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:08:14 2022] DELETE /identity/v3/users/baf607bf43c84c579b275a23507438a4 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:15.463316 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276c8ac-9421-4f15-b56f-3d72dc6c2583 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:15.466122 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1276c8ac-9421-4f15-b56f-3d72dc6c2583 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:15.506630 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1276c8ac-9421-4f15-b56f-3d72dc6c2583 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:15.624170 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56082df-35ee-442c-b436-e3ecb4161ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:15.625728 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56082df-35ee-442c-b436-e3ecb4161ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:15.626835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56082df-35ee-442c-b436-e3ecb4161ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:15.627842 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56082df-35ee-442c-b436-e3ecb4161ecc admin admin] PATH_INFO: `/v3/projects/f61ec7306b9b457a9d1b4a5e32e2a2e4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:15.629998 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56082df-35ee-442c-b436-e3ecb4161ecc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f61ec7306b9b457a9d1b4a5e32e2a2e4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:15.633094 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56082df-35ee-442c-b436-e3ecb4161ecc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:15.852116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:15.862076 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f56082df-35ee-442c-b436-e3ecb4161ecc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f61ec7306b9b457a9d1b4a5e32e2a2e4', 'request_id': 'req-f56082df-35ee-442c-b436-e3ecb4161ecc'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:15.868742 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f56082df-35ee-442c-b436-e3ecb4161ecc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f61ec7306b9b457a9d1b4a5e32e2a2e4', 'request_id': 'req-f56082df-35ee-442c-b436-e3ecb4161ecc'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:15.877156 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f56082df-35ee-442c-b436-e3ecb4161ecc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f61ec7306b9b457a9d1b4a5e32e2a2e4', 'request_id': 'req-f56082df-35ee-442c-b436-e3ecb4161ecc'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:15.969896 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f56082df-35ee-442c-b436-e3ecb4161ecc admin admin] The token cache is being invalidate because project f61ec7306b9b457a9d1b4a5e32e2a2e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:08:15.969896 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f56082df-35ee-442c-b436-e3ecb4161ecc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:15.969896 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f56082df-35ee-442c-b436-e3ecb4161ecc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:15.972083 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 988/1982] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:08:15 2022] DELETE /identity/v3/projects/f61ec7306b9b457a9d1b4a5e32e2a2e4 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:15.976790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a4cf9-dea2-4910-8589-57c7baaf3ca1 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:15.977076 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a4cf9-dea2-4910-8589-57c7baaf3ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:15.977352 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669a4cf9-dea2-4910-8589-57c7baaf3ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:15.999845 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-669a4cf9-dea2-4910-8589-57c7baaf3ca1 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:15.999845 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-669a4cf9-dea2-4910-8589-57c7baaf3ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:16.094885 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 995/1983] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:16.103485 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ce21e-cb5d-4dc1-b055-9545ae3373af None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:16.104075 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ce21e-cb5d-4dc1-b055-9545ae3373af None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:16.104579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ce21e-cb5d-4dc1-b055-9545ae3373af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:16.148560 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6e5ce21e-cb5d-4dc1-b055-9545ae3373af None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:16.149903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6e5ce21e-cb5d-4dc1-b055-9545ae3373af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:16.165266 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 989/1984] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:16.167391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669a4cf9-dea2-4910-8589-57c7baaf3ca1 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:16.185058 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669a4cf9-dea2-4910-8589-57c7baaf3ca1 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:16.193879 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377962f0-29d0-49c3-a3c4-1d557021ba21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:16.202209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377962f0-29d0-49c3-a3c4-1d557021ba21 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:16.202569 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377962f0-29d0-49c3-a3c4-1d557021ba21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:16.202979 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377962f0-29d0-49c3-a3c4-1d557021ba21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:16.203779 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377962f0-29d0-49c3-a3c4-1d557021ba21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:08:16.204357 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377962f0-29d0-49c3-a3c4-1d557021ba21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:16.206107 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377962f0-29d0-49c3-a3c4-1d557021ba21 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:16.218370 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 996/1985] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 20:08:16.223025 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:DELETE=13,keystone:SELECT=83,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:16.224344 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ce21e-cb5d-4dc1-b055-9545ae3373af None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:16.236123 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ce21e-cb5d-4dc1-b055-9545ae3373af None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:16.241256 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d8ed4b-5aab-49a2-ab36-2ac383d0162b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:16.241854 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8ed4b-5aab-49a2-ab36-2ac383d0162b admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:16.242305 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8ed4b-5aab-49a2-ab36-2ac383d0162b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:16.242730 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8ed4b-5aab-49a2-ab36-2ac383d0162b admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:16.243671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d8ed4b-5aab-49a2-ab36-2ac383d0162b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:16.246504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d8ed4b-5aab-49a2-ab36-2ac383d0162b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:16.367894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 990/1986] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:08:16 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:16.372165 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377962f0-29d0-49c3-a3c4-1d557021ba21 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:16.383202 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377962f0-29d0-49c3-a3c4-1d557021ba21 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:16.444194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca7996-0764-4437-8274-4e19d8b7dee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:16.445146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca7996-0764-4437-8274-4e19d8b7dee1 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:16.445627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca7996-0764-4437-8274-4e19d8b7dee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:16.445999 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca7996-0764-4437-8274-4e19d8b7dee1 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:16.447068 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca7996-0764-4437-8274-4e19d8b7dee1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:16.449179 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca7996-0764-4437-8274-4e19d8b7dee1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:16.615161 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 997/1987] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:08:16 2022] POST /identity/v3/users => generated 287 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:16.618085 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d8ed4b-5aab-49a2-ab36-2ac383d0162b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:16.630710 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d8ed4b-5aab-49a2-ab36-2ac383d0162b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:16.637323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996a635-c71f-4c56-9c86-f835cb4ad3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:16.637958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996a635-c71f-4c56-9c86-f835cb4ad3ed admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:16.638221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996a635-c71f-4c56-9c86-f835cb4ad3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:16.638490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996a635-c71f-4c56-9c86-f835cb4ad3ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:16.639404 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0996a635-c71f-4c56-9c86-f835cb4ad3ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:16.641306 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0996a635-c71f-4c56-9c86-f835cb4ad3ed admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:16.652742 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 991/1988] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:16.656473 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca7996-0764-4437-8274-4e19d8b7dee1 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:16.667399 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca7996-0764-4437-8274-4e19d8b7dee1 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:16.675704 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b900d-0303-4682-a8f3-8de3917d818a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:16.689434 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b900d-0303-4682-a8f3-8de3917d818a admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:16.690025 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b900d-0303-4682-a8f3-8de3917d818a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:16.690333 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b900d-0303-4682-a8f3-8de3917d818a admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:16.691099 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b900d-0303-4682-a8f3-8de3917d818a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:16.698586 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b900d-0303-4682-a8f3-8de3917d818a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:16.708368 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 998/1989] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:16.712619 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996a635-c71f-4c56-9c86-f835cb4ad3ed admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:16.723290 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996a635-c71f-4c56-9c86-f835cb4ad3ed admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:16.728279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87d316-6420-442c-be62-00a5a35e9491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:16.733478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87d316-6420-442c-be62-00a5a35e9491 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:16.733766 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87d316-6420-442c-be62-00a5a35e9491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:16.734055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c87d316-6420-442c-be62-00a5a35e9491 admin admin] PATH_INFO: `/v3/projects/acdca1f9399e4dc7bdffc6300ce819da/users/3776cafbc946488299a680c8c47d2eed/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:16.775219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c87d316-6420-442c-be62-00a5a35e9491 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acdca1f9399e4dc7bdffc6300ce819da, user_id=3776cafbc946488299a680c8c47d2eed, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:16.778409 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c87d316-6420-442c-be62-00a5a35e9491 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:16.840533 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 992/1990] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:08:16 2022] PUT /identity/v3/projects/acdca1f9399e4dc7bdffc6300ce819da/users/3776cafbc946488299a680c8c47d2eed/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:08:16.852556 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59db881-adb4-498d-9879-7a3835dfd7fb None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:16.853101 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59db881-adb4-498d-9879-7a3835dfd7fb None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:16.853203 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59db881-adb4-498d-9879-7a3835dfd7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:16.882127 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c59db881-adb4-498d-9879-7a3835dfd7fb None None] MFA Rules not processed for user `3776cafbc946488299a680c8c47d2eed`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:16.883398 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c59db881-adb4-498d-9879-7a3835dfd7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:16.954263 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 999/1991] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:16.958893 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87d316-6420-442c-be62-00a5a35e9491 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:17.023321 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c87d316-6420-442c-be62-00a5a35e9491 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:17.027944 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b241d62-b1b9-4199-9566-8601ed950e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:17.028633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b241d62-b1b9-4199-9566-8601ed950e62 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:17.028933 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b241d62-b1b9-4199-9566-8601ed950e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:17.029238 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b241d62-b1b9-4199-9566-8601ed950e62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:17.029940 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b241d62-b1b9-4199-9566-8601ed950e62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:17.032475 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b241d62-b1b9-4199-9566-8601ed950e62 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:17.050696 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:17.115826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 993/1992] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:08:16 2022] POST /identity/v3/projects => generated 375 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:17.119978 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59db881-adb4-498d-9879-7a3835dfd7fb None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:17.131753 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59db881-adb4-498d-9879-7a3835dfd7fb None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:17.189636 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648c621-4fc6-4b0e-a908-4c3e181b3d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:17.190343 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648c621-4fc6-4b0e-a908-4c3e181b3d9b admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:17.190624 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648c621-4fc6-4b0e-a908-4c3e181b3d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:17.190896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648c621-4fc6-4b0e-a908-4c3e181b3d9b admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:17.191755 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3648c621-4fc6-4b0e-a908-4c3e181b3d9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:17.193808 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3648c621-4fc6-4b0e-a908-4c3e181b3d9b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:17.280916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1000/1993] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:08:17 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:08:17.284797 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b241d62-b1b9-4199-9566-8601ed950e62 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:17.296276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b241d62-b1b9-4199-9566-8601ed950e62 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:17.301383 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df4ab8-5ca3-4a8b-9878-648b82c96433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:17.302057 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df4ab8-5ca3-4a8b-9878-648b82c96433 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:17.302651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df4ab8-5ca3-4a8b-9878-648b82c96433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:17.302651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df4ab8-5ca3-4a8b-9878-648b82c96433 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:17.303570 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61df4ab8-5ca3-4a8b-9878-648b82c96433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:17.305217 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61df4ab8-5ca3-4a8b-9878-648b82c96433 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:17.320240 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 994/1994] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:17.322504 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648c621-4fc6-4b0e-a908-4c3e181b3d9b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:17.332396 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:17.336569 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3648c621-4fc6-4b0e-a908-4c3e181b3d9b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:17.349881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17998e9-a299-4b4b-a840-d894008ef904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:17.350795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17998e9-a299-4b4b-a840-d894008ef904 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:17.351095 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17998e9-a299-4b4b-a840-d894008ef904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:17.351369 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17998e9-a299-4b4b-a840-d894008ef904 admin admin] PATH_INFO: `/v3/projects/c786b400270d44c2b6ea0b56727d5b0b/users/6ae9a59f5e6047539581593112e711b9/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:17.367992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17998e9-a299-4b4b-a840-d894008ef904 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c786b400270d44c2b6ea0b56727d5b0b, user_id=6ae9a59f5e6047539581593112e711b9, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:17.370750 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17998e9-a299-4b4b-a840-d894008ef904 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:17.429958 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1001/1995] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:08:17 2022] PUT /identity/v3/projects/c786b400270d44c2b6ea0b56727d5b0b/users/6ae9a59f5e6047539581593112e711b9/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:17.433980 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df4ab8-5ca3-4a8b-9878-648b82c96433 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:17.444419 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df4ab8-5ca3-4a8b-9878-648b82c96433 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:17.499497 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1280dacb-9a90-4183-a0b8-c11b5793dda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:17.500198 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280dacb-9a90-4183-a0b8-c11b5793dda9 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:17.500507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280dacb-9a90-4183-a0b8-c11b5793dda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:17.500784 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280dacb-9a90-4183-a0b8-c11b5793dda9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:17.501507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1280dacb-9a90-4183-a0b8-c11b5793dda9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:17.503723 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1280dacb-9a90-4183-a0b8-c11b5793dda9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:17.510403 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:17.515366 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 995/1996] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:17.520334 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17998e9-a299-4b4b-a840-d894008ef904 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:17.530477 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17998e9-a299-4b4b-a840-d894008ef904 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:17.535122 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23ae07-6a48-4184-9b35-619f5d2108ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:17.536029 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23ae07-6a48-4184-9b35-619f5d2108ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:17.536279 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23ae07-6a48-4184-9b35-619f5d2108ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:17.536575 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23ae07-6a48-4184-9b35-619f5d2108ac admin admin] PATH_INFO: `/v3/projects/c786b400270d44c2b6ea0b56727d5b0b/users/6ae9a59f5e6047539581593112e711b9/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:17.538572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed23ae07-6a48-4184-9b35-619f5d2108ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=c786b400270d44c2b6ea0b56727d5b0b, user_id=6ae9a59f5e6047539581593112e711b9, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:17.541322 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed23ae07-6a48-4184-9b35-619f5d2108ac admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:17.551770 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1002/1997] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:08:17 2022] PUT /identity/v3/projects/c786b400270d44c2b6ea0b56727d5b0b/users/6ae9a59f5e6047539581593112e711b9/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:17.561978 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119f37e-d7c8-43b9-8c36-9b2384d7faa1 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:17.562264 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119f37e-d7c8-43b9-8c36-9b2384d7faa1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:17.562542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119f37e-d7c8-43b9-8c36-9b2384d7faa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:17.571373 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:17.589069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8119f37e-d7c8-43b9-8c36-9b2384d7faa1 None None] MFA Rules not processed for user `6ae9a59f5e6047539581593112e711b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:17.590289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8119f37e-d7c8-43b9-8c36-9b2384d7faa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:17.655450 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 996/1998] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:17 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:17.666219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23ae07-6a48-4184-9b35-619f5d2108ac admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:17.667913 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ed23ae07-6a48-4184-9b35-619f5d2108ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:17.727377 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23ae07-6a48-4184-9b35-619f5d2108ac admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:17.731234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c3fe6-91bf-4ad8-9038-27cd497fbcde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:17.731892 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c3fe6-91bf-4ad8-9038-27cd497fbcde service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:17.732177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c3fe6-91bf-4ad8-9038-27cd497fbcde service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:17.732487 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c3fe6-91bf-4ad8-9038-27cd497fbcde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:17.793492 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c3fe6-91bf-4ad8-9038-27cd497fbcde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:17.794948 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c3fe6-91bf-4ad8-9038-27cd497fbcde service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:17.804497 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1003/1999] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:08:17.851295 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:17.897556 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:18.543015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:19.211356 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:19.651333 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=86,keystone:INSERT=5,keystone:DELETE=9 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:20.662707 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:20.931689 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:21.123443 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8119f37e-d7c8-43b9-8c36-9b2384d7faa1 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:21.201929 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8119f37e-d7c8-43b9-8c36-9b2384d7faa1 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:21.207187 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6be8c-a3bb-4caa-8495-af34318c2b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:21.211020 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6be8c-a3bb-4caa-8495-af34318c2b7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:21.211413 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6be8c-a3bb-4caa-8495-af34318c2b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:21.211811 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6be8c-a3bb-4caa-8495-af34318c2b7b admin admin] PATH_INFO: `/v3/users/3776cafbc946488299a680c8c47d2eed` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:21.213700 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a6be8c-a3bb-4caa-8495-af34318c2b7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3776cafbc946488299a680c8c47d2eed)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:21.215847 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a6be8c-a3bb-4caa-8495-af34318c2b7b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:21.428654 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-22a6be8c-a3bb-4caa-8495-af34318c2b7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3776cafbc946488299a680c8c47d2eed'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:21.429318 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-22a6be8c-a3bb-4caa-8495-af34318c2b7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3776cafbc946488299a680c8c47d2eed'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:21.443815 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-22a6be8c-a3bb-4caa-8495-af34318c2b7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3776cafbc946488299a680c8c47d2eed'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:21.449629 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-22a6be8c-a3bb-4caa-8495-af34318c2b7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3776cafbc946488299a680c8c47d2eed'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:21.513743 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-22a6be8c-a3bb-4caa-8495-af34318c2b7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3776cafbc946488299a680c8c47d2eed'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:21.525709 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 997/2000] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:08:21 2022] DELETE /identity/v3/users/3776cafbc946488299a680c8c47d2eed => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:21.537676 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91019234-64f4-42f4-94e4-18cc27c6ce29 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:21.538061 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91019234-64f4-42f4-94e4-18cc27c6ce29 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:21.538392 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91019234-64f4-42f4-94e4-18cc27c6ce29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:21.563014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-91019234-64f4-42f4-94e4-18cc27c6ce29 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:21.565612 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-91019234-64f4-42f4-94e4-18cc27c6ce29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:21.644959 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1004/2001] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:21.653722 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6be8c-a3bb-4caa-8495-af34318c2b7b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:21.655938 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-22a6be8c-a3bb-4caa-8495-af34318c2b7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:21.724411 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a6be8c-a3bb-4caa-8495-af34318c2b7b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:21.727915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360503d-18f7-46f7-8615-d1cb5b05010a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:21.728603 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360503d-18f7-46f7-8615-d1cb5b05010a service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:21.728891 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360503d-18f7-46f7-8615-d1cb5b05010a service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:21.729209 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360503d-18f7-46f7-8615-d1cb5b05010a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:21.738522 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7360503d-18f7-46f7-8615-d1cb5b05010a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:21.740351 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7360503d-18f7-46f7-8615-d1cb5b05010a service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:21.749974 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 998/2002] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:08:21.783032 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:21.942872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:22.255838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619c81f-4cf8-4025-b470-8e6381f879ac None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:22.256339 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619c81f-4cf8-4025-b470-8e6381f879ac None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:22.256779 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2619c81f-4cf8-4025-b470-8e6381f879ac None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:22.258297 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1005/2003] 10.222.0.44 () {54 vars in 921 bytes} [Thu Jun 23 20:08:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:08:22.283314 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42667f2a-5244-4f57-bca0-5cd5830a2d9b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:22.284130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42667f2a-5244-4f57-bca0-5cd5830a2d9b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:22.284459 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42667f2a-5244-4f57-bca0-5cd5830a2d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:22.306324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-42667f2a-5244-4f57-bca0-5cd5830a2d9b None None] MFA Rules not processed for user `9715d5deeccd419b97cb0bd39e17982f`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:22.307573 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-42667f2a-5244-4f57-bca0-5cd5830a2d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:22.374788 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 999/2004] 10.222.0.44 () {58 vars in 1018 bytes} [Thu Jun 23 20:08:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:22.398020 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619c81f-4cf8-4025-b470-8e6381f879ac None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:22.400029 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2619c81f-4cf8-4025-b470-8e6381f879ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:22.418039 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2619c81f-4cf8-4025-b470-8e6381f879ac None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:22.425350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d434657-c491-4036-a8d6-54885412a0f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9715d5deeccd419b97cb0bd39e17982f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:22.426102 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d434657-c491-4036-a8d6-54885412a0f9 service placement] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:22.426470 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d434657-c491-4036-a8d6-54885412a0f9 service placement] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:22.427037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d434657-c491-4036-a8d6-54885412a0f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:22.436762 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d434657-c491-4036-a8d6-54885412a0f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:22.438657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d434657-c491-4036-a8d6-54885412a0f9 service placement] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:22.449830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1006/2005] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:08:22.602415 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42667f2a-5244-4f57-bca0-5cd5830a2d9b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:22.604114 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-42667f2a-5244-4f57-bca0-5cd5830a2d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:22.622775 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42667f2a-5244-4f57-bca0-5cd5830a2d9b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:22.625772 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41729b-3fac-4b14-8867-fa53ba3e02ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:22.626701 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41729b-3fac-4b14-8867-fa53ba3e02ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:22.626979 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41729b-3fac-4b14-8867-fa53ba3e02ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:22.627463 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41729b-3fac-4b14-8867-fa53ba3e02ed admin admin] PATH_INFO: `/v3/projects/acdca1f9399e4dc7bdffc6300ce819da` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:22.628703 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf41729b-3fac-4b14-8867-fa53ba3e02ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=acdca1f9399e4dc7bdffc6300ce819da)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:22.630643 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf41729b-3fac-4b14-8867-fa53ba3e02ed admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:22.775020 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cf41729b-3fac-4b14-8867-fa53ba3e02ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acdca1f9399e4dc7bdffc6300ce819da', 'request_id': 'req-cf41729b-3fac-4b14-8867-fa53ba3e02ed'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:22.781468 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cf41729b-3fac-4b14-8867-fa53ba3e02ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acdca1f9399e4dc7bdffc6300ce819da', 'request_id': 'req-cf41729b-3fac-4b14-8867-fa53ba3e02ed'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:22.788693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cf41729b-3fac-4b14-8867-fa53ba3e02ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acdca1f9399e4dc7bdffc6300ce819da', 'request_id': 'req-cf41729b-3fac-4b14-8867-fa53ba3e02ed'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:22.862388 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cf41729b-3fac-4b14-8867-fa53ba3e02ed admin admin] The token cache is being invalidate because project acdca1f9399e4dc7bdffc6300ce819da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:08:22.863786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cf41729b-3fac-4b14-8867-fa53ba3e02ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:22.863786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cf41729b-3fac-4b14-8867-fa53ba3e02ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:22.864953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1000/2006] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:08:22 2022] DELETE /identity/v3/projects/acdca1f9399e4dc7bdffc6300ce819da => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:22.868815 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d434657-c491-4036-a8d6-54885412a0f9 service placement] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:22.880609 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d434657-c491-4036-a8d6-54885412a0f9 service placement] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:22.944092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7fc1b8-a848-4d24-9917-3f7fbbea2e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:22.945530 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7fc1b8-a848-4d24-9917-3f7fbbea2e05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:22.945826 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7fc1b8-a848-4d24-9917-3f7fbbea2e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:22.946132 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7fc1b8-a848-4d24-9917-3f7fbbea2e05 admin admin] PATH_INFO: `/v3/users/6ae9a59f5e6047539581593112e711b9` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:22.947672 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7fc1b8-a848-4d24-9917-3f7fbbea2e05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ae9a59f5e6047539581593112e711b9)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:22.949657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7fc1b8-a848-4d24-9917-3f7fbbea2e05 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:23.105806 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983dd26c-4008-458c-bad0-9e46e682b7a5 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:23.106336 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983dd26c-4008-458c-bad0-9e46e682b7a5 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:23.106439 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983dd26c-4008-458c-bad0-9e46e682b7a5 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:23.107808 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1001/2007] 10.222.0.44 () {54 vars in 921 bytes} [Thu Jun 23 20:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:08:23.114030 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeea055-4ed7-4440-8a9e-97df7e54c3dc None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:23.114317 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeea055-4ed7-4440-8a9e-97df7e54c3dc None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:23.114974 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeea055-4ed7-4440-8a9e-97df7e54c3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:23.138504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-fbeea055-4ed7-4440-8a9e-97df7e54c3dc None None] MFA Rules not processed for user `9715d5deeccd419b97cb0bd39e17982f`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:23.138504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fbeea055-4ed7-4440-8a9e-97df7e54c3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:23.201887 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1002/2008] 10.222.0.44 () {58 vars in 1018 bytes} [Thu Jun 23 20:08:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:23.202830 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeea055-4ed7-4440-8a9e-97df7e54c3dc None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:23.204207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fbeea055-4ed7-4440-8a9e-97df7e54c3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:23.204854 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-da7fc1b8-a848-4d24-9917-3f7fbbea2e05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ae9a59f5e6047539581593112e711b9'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:23.205695 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-da7fc1b8-a848-4d24-9917-3f7fbbea2e05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ae9a59f5e6047539581593112e711b9'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:23.224061 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeea055-4ed7-4440-8a9e-97df7e54c3dc None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:23.234130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa526be8-f428-430b-afe2-eb1cf328730b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:23.234976 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa526be8-f428-430b-afe2-eb1cf328730b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:23.235318 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa526be8-f428-430b-afe2-eb1cf328730b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:23.235600 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa526be8-f428-430b-afe2-eb1cf328730b admin admin] PATH_INFO: `/v3/users/998bbc74a41b47e6a561ce97c4d281b6` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:23.236990 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa526be8-f428-430b-afe2-eb1cf328730b admin admin] RBAC: Authorizing `identity:delete_user(user_id=998bbc74a41b47e6a561ce97c4d281b6)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:23.241433 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa526be8-f428-430b-afe2-eb1cf328730b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:23.296493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-da7fc1b8-a848-4d24-9917-3f7fbbea2e05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ae9a59f5e6047539581593112e711b9'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:23.297293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-da7fc1b8-a848-4d24-9917-3f7fbbea2e05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ae9a59f5e6047539581593112e711b9'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:23.310297 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:23.312636 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-da7fc1b8-a848-4d24-9917-3f7fbbea2e05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ae9a59f5e6047539581593112e711b9'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:23.325454 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1007/2009] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:08:22 2022] DELETE /identity/v3/users/6ae9a59f5e6047539581593112e711b9 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:08:23.328447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7fc1b8-a848-4d24-9917-3f7fbbea2e05 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:23.328447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-da7fc1b8-a848-4d24-9917-3f7fbbea2e05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:23.401683 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7fc1b8-a848-4d24-9917-3f7fbbea2e05 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:23.406643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8e0d8-3704-4e22-9154-07e8135fd119 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9715d5deeccd419b97cb0bd39e17982f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:23.407295 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8e0d8-3704-4e22-9154-07e8135fd119 service placement] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:23.407574 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8e0d8-3704-4e22-9154-07e8135fd119 service placement] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:23.407873 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8e0d8-3704-4e22-9154-07e8135fd119 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:23.409748 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-54c8e0d8-3704-4e22-9154-07e8135fd119 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:23.421206 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c8e0d8-3704-4e22-9154-07e8135fd119 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:23.422870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c8e0d8-3704-4e22-9154-07e8135fd119 service placement] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:23.433417 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1008/2010] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:08:23.434383 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8e0d8-3704-4e22-9154-07e8135fd119 service placement] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:23.435827 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-54c8e0d8-3704-4e22-9154-07e8135fd119 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:23.481861 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fa526be8-f428-430b-afe2-eb1cf328730b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '998bbc74a41b47e6a561ce97c4d281b6'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:23.482129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fa526be8-f428-430b-afe2-eb1cf328730b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '998bbc74a41b47e6a561ce97c4d281b6'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:23.494075 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:23.516740 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8e0d8-3704-4e22-9154-07e8135fd119 service placement] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:23.519636 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fa526be8-f428-430b-afe2-eb1cf328730b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '998bbc74a41b47e6a561ce97c4d281b6'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:23.522750 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25192b0e-dd46-4f9e-8a27-e9b0104198a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:23.523880 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25192b0e-dd46-4f9e-8a27-e9b0104198a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:23.524316 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25192b0e-dd46-4f9e-8a27-e9b0104198a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:23.524702 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25192b0e-dd46-4f9e-8a27-e9b0104198a3 admin admin] PATH_INFO: `/v3/users/c41d51884a3b47318c6d05fba5918fa7` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:23.526118 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25192b0e-dd46-4f9e-8a27-e9b0104198a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c41d51884a3b47318c6d05fba5918fa7)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:23.528267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25192b0e-dd46-4f9e-8a27-e9b0104198a3 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:23.538338 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fa526be8-f428-430b-afe2-eb1cf328730b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '998bbc74a41b47e6a561ce97c4d281b6'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:23.616414 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fa526be8-f428-430b-afe2-eb1cf328730b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '998bbc74a41b47e6a561ce97c4d281b6'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:23.622924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1003/2011] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:08:23 2022] DELETE /identity/v3/users/998bbc74a41b47e6a561ce97c4d281b6 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:23.627834 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b54b75-2943-4e90-b374-3251a77b2ae1 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:23.628197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b54b75-2943-4e90-b374-3251a77b2ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:23.628569 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b54b75-2943-4e90-b374-3251a77b2ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:23.647227 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-34b54b75-2943-4e90-b374-3251a77b2ae1 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:23.651824 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-34b54b75-2943-4e90-b374-3251a77b2ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:23.714222 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1004/2012] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:08:23.724323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b54b75-2943-4e90-b374-3251a77b2ae1 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:23.726294 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-34b54b75-2943-4e90-b374-3251a77b2ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:23.742489 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-25192b0e-dd46-4f9e-8a27-e9b0104198a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c41d51884a3b47318c6d05fba5918fa7'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:23.743510 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-25192b0e-dd46-4f9e-8a27-e9b0104198a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c41d51884a3b47318c6d05fba5918fa7'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:23.805927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b54b75-2943-4e90-b374-3251a77b2ae1 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:23.809706 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a5c06-77f5-4794-ae23-b5f035cc687e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:23.814531 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a5c06-77f5-4794-ae23-b5f035cc687e service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:23.814531 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a5c06-77f5-4794-ae23-b5f035cc687e service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:23.814531 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a5c06-77f5-4794-ae23-b5f035cc687e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:23.821390 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a5c06-77f5-4794-ae23-b5f035cc687e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:23.823118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a5c06-77f5-4794-ae23-b5f035cc687e service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:23.832686 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1005/2013] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 20:08:23.834595 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-25192b0e-dd46-4f9e-8a27-e9b0104198a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c41d51884a3b47318c6d05fba5918fa7'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:23.835233 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-25192b0e-dd46-4f9e-8a27-e9b0104198a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c41d51884a3b47318c6d05fba5918fa7'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:23.848182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-25192b0e-dd46-4f9e-8a27-e9b0104198a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c41d51884a3b47318c6d05fba5918fa7'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:23.860320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1009/2014] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:08:23 2022] DELETE /identity/v3/users/c41d51884a3b47318c6d05fba5918fa7 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:23.947682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a5c06-77f5-4794-ae23-b5f035cc687e service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:23.949179 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2c2a5c06-77f5-4794-ae23-b5f035cc687e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:23.967930 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a5c06-77f5-4794-ae23-b5f035cc687e service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:24.039944 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63e716-d29d-4446-a456-f45f3204f733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:24.040841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63e716-d29d-4446-a456-f45f3204f733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:24.041117 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63e716-d29d-4446-a456-f45f3204f733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:24.041384 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63e716-d29d-4446-a456-f45f3204f733 admin admin] PATH_INFO: `/v3/projects/c786b400270d44c2b6ea0b56727d5b0b` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:24.042566 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63e716-d29d-4446-a456-f45f3204f733 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c786b400270d44c2b6ea0b56727d5b0b)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:24.044775 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63e716-d29d-4446-a456-f45f3204f733 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:24.179777 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25192b0e-dd46-4f9e-8a27-e9b0104198a3 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:24.182469 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-25192b0e-dd46-4f9e-8a27-e9b0104198a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:24.198087 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25192b0e-dd46-4f9e-8a27-e9b0104198a3 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:24.205462 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14338e35-84ec-4d80-ab15-2925e07d7d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:24.209773 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14338e35-84ec-4d80-ab15-2925e07d7d38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:24.209773 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14338e35-84ec-4d80-ab15-2925e07d7d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:24.209773 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14338e35-84ec-4d80-ab15-2925e07d7d38 admin admin] PATH_INFO: `/v3/projects/ce4e3f14283b45619321876b00b4d466` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:24.209773 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14338e35-84ec-4d80-ab15-2925e07d7d38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce4e3f14283b45619321876b00b4d466)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:24.210841 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14338e35-84ec-4d80-ab15-2925e07d7d38 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:24.224499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cf63e716-d29d-4446-a456-f45f3204f733 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c786b400270d44c2b6ea0b56727d5b0b', 'request_id': 'req-cf63e716-d29d-4446-a456-f45f3204f733'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:24.245674 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cf63e716-d29d-4446-a456-f45f3204f733 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c786b400270d44c2b6ea0b56727d5b0b', 'request_id': 'req-cf63e716-d29d-4446-a456-f45f3204f733'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:24.251976 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cf63e716-d29d-4446-a456-f45f3204f733 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c786b400270d44c2b6ea0b56727d5b0b', 'request_id': 'req-cf63e716-d29d-4446-a456-f45f3204f733'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:24.353314 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cf63e716-d29d-4446-a456-f45f3204f733 admin admin] The token cache is being invalidate because project c786b400270d44c2b6ea0b56727d5b0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:08:24.353536 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cf63e716-d29d-4446-a456-f45f3204f733 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:24.354176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cf63e716-d29d-4446-a456-f45f3204f733 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:24.355758 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1006/2015] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:08:23 2022] DELETE /identity/v3/projects/c786b400270d44c2b6ea0b56727d5b0b => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:24.361444 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1445d-94bd-47dd-97e8-746a9fc1986e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:24.361739 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1445d-94bd-47dd-97e8-746a9fc1986e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:24.362022 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1445d-94bd-47dd-97e8-746a9fc1986e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:24.386076 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b5d1445d-94bd-47dd-97e8-746a9fc1986e None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:24.387567 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b5d1445d-94bd-47dd-97e8-746a9fc1986e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:24.395176 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-14338e35-84ec-4d80-ab15-2925e07d7d38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce4e3f14283b45619321876b00b4d466', 'request_id': 'req-14338e35-84ec-4d80-ab15-2925e07d7d38'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:24.451243 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:24.453606 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1007/2016] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:08:24.454560 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1445d-94bd-47dd-97e8-746a9fc1986e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:24.455861 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b5d1445d-94bd-47dd-97e8-746a9fc1986e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:24.467516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1445d-94bd-47dd-97e8-746a9fc1986e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:24.473989 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372cc1c-5bcf-4450-a5ea-d3b2ba0662d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:24.473989 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372cc1c-5bcf-4450-a5ea-d3b2ba0662d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:24.473989 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372cc1c-5bcf-4450-a5ea-d3b2ba0662d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:24.473989 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372cc1c-5bcf-4450-a5ea-d3b2ba0662d2 admin admin] PATH_INFO: `/v3/projects/c549ea77816d4fa88079e9087caef36e` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:24.475163 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372cc1c-5bcf-4450-a5ea-d3b2ba0662d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c549ea77816d4fa88079e9087caef36e)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:24.475874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-14338e35-84ec-4d80-ab15-2925e07d7d38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce4e3f14283b45619321876b00b4d466', 'request_id': 'req-14338e35-84ec-4d80-ab15-2925e07d7d38'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:24.477361 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372cc1c-5bcf-4450-a5ea-d3b2ba0662d2 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:24.483387 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-14338e35-84ec-4d80-ab15-2925e07d7d38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce4e3f14283b45619321876b00b4d466', 'request_id': 'req-14338e35-84ec-4d80-ab15-2925e07d7d38'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:24.496546 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-14338e35-84ec-4d80-ab15-2925e07d7d38 admin admin] The token cache is being invalidate because project ce4e3f14283b45619321876b00b4d466 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:08:24.496832 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-14338e35-84ec-4d80-ab15-2925e07d7d38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:24.497507 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-14338e35-84ec-4d80-ab15-2925e07d7d38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:24.499089 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1010/2017] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:08:24 2022] DELETE /identity/v3/projects/ce4e3f14283b45619321876b00b4d466 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:08:24.502656 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdeb8b1-ff8b-496d-9828-1deba65cb95c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:24.502656 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdeb8b1-ff8b-496d-9828-1deba65cb95c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:24.502656 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdeb8b1-ff8b-496d-9828-1deba65cb95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:24.520337 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-0cdeb8b1-ff8b-496d-9828-1deba65cb95c None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:24.521539 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0cdeb8b1-ff8b-496d-9828-1deba65cb95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:24.530535 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1011/2018] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:24.531453 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdeb8b1-ff8b-496d-9828-1deba65cb95c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:24.532749 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0cdeb8b1-ff8b-496d-9828-1deba65cb95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:24.544154 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdeb8b1-ff8b-496d-9828-1deba65cb95c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:24.549687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45982e-3a94-4f70-af8d-992e3521da0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:24.550582 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45982e-3a94-4f70-af8d-992e3521da0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:24.550865 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45982e-3a94-4f70-af8d-992e3521da0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:24.551139 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45982e-3a94-4f70-af8d-992e3521da0b admin admin] PATH_INFO: `/v3/users/48b728a8c896459d830bd07be28d6f02` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:24.552364 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45982e-3a94-4f70-af8d-992e3521da0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=48b728a8c896459d830bd07be28d6f02)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:24.554296 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45982e-3a94-4f70-af8d-992e3521da0b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:24.588159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:24.641980 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9372cc1c-5bcf-4450-a5ea-d3b2ba0662d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c549ea77816d4fa88079e9087caef36e', 'request_id': 'req-9372cc1c-5bcf-4450-a5ea-d3b2ba0662d2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:24.648111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9372cc1c-5bcf-4450-a5ea-d3b2ba0662d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c549ea77816d4fa88079e9087caef36e', 'request_id': 'req-9372cc1c-5bcf-4450-a5ea-d3b2ba0662d2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:24.653474 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9372cc1c-5bcf-4450-a5ea-d3b2ba0662d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c549ea77816d4fa88079e9087caef36e', 'request_id': 'req-9372cc1c-5bcf-4450-a5ea-d3b2ba0662d2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:24.776774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9372cc1c-5bcf-4450-a5ea-d3b2ba0662d2 admin admin] The token cache is being invalidate because project c549ea77816d4fa88079e9087caef36e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:08:24.777705 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9372cc1c-5bcf-4450-a5ea-d3b2ba0662d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:24.777824 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9372cc1c-5bcf-4450-a5ea-d3b2ba0662d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:24.779895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1008/2019] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:08:24 2022] DELETE /identity/v3/projects/c549ea77816d4fa88079e9087caef36e => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:24.784424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372cc1c-5bcf-4450-a5ea-d3b2ba0662d2 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:24.786051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9372cc1c-5bcf-4450-a5ea-d3b2ba0662d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:24.805053 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0e45982e-3a94-4f70-af8d-992e3521da0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48b728a8c896459d830bd07be28d6f02'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:24.806348 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0e45982e-3a94-4f70-af8d-992e3521da0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48b728a8c896459d830bd07be28d6f02'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:24.861947 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372cc1c-5bcf-4450-a5ea-d3b2ba0662d2 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:24.868823 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90725e3c-7780-41e5-91aa-e58cc12cce2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:24.869508 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90725e3c-7780-41e5-91aa-e58cc12cce2f admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:24.869794 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90725e3c-7780-41e5-91aa-e58cc12cce2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:24.870066 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90725e3c-7780-41e5-91aa-e58cc12cce2f admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:24.870810 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90725e3c-7780-41e5-91aa-e58cc12cce2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:08:24.871120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90725e3c-7780-41e5-91aa-e58cc12cce2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:24.872858 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90725e3c-7780-41e5-91aa-e58cc12cce2f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:24.888599 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1009/2020] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:08:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:08:24.888599 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90725e3c-7780-41e5-91aa-e58cc12cce2f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:24.888599 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-90725e3c-7780-41e5-91aa-e58cc12cce2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:24.901541 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90725e3c-7780-41e5-91aa-e58cc12cce2f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:24.906351 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e7504-36c1-48c5-8235-bcb01838e3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:24.907228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e7504-36c1-48c5-8235-bcb01838e3e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:24.908107 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e7504-36c1-48c5-8235-bcb01838e3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:24.908432 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89e7504-36c1-48c5-8235-bcb01838e3e3 admin admin] PATH_INFO: `/v3/users/dc1da801397c48a2831d83bc2f0ece0c` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:24.909685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e7504-36c1-48c5-8235-bcb01838e3e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc1da801397c48a2831d83bc2f0ece0c)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:24.911637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89e7504-36c1-48c5-8235-bcb01838e3e3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:24.927428 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0e45982e-3a94-4f70-af8d-992e3521da0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48b728a8c896459d830bd07be28d6f02'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:24.936419 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0e45982e-3a94-4f70-af8d-992e3521da0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48b728a8c896459d830bd07be28d6f02'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:24.943420 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0e45982e-3a94-4f70-af8d-992e3521da0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48b728a8c896459d830bd07be28d6f02'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:24.959290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1012/2021] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:08:24 2022] DELETE /identity/v3/users/48b728a8c896459d830bd07be28d6f02 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:24.960072 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45982e-3a94-4f70-af8d-992e3521da0b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:24.961444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0e45982e-3a94-4f70-af8d-992e3521da0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:25.043068 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45982e-3a94-4f70-af8d-992e3521da0b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:25.049140 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33f9c6-b568-4e06-a85c-778e6f2a7543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:25.049819 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33f9c6-b568-4e06-a85c-778e6f2a7543 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:25.050103 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33f9c6-b568-4e06-a85c-778e6f2a7543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:25.050376 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33f9c6-b568-4e06-a85c-778e6f2a7543 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:25.051079 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33f9c6-b568-4e06-a85c-778e6f2a7543 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:25.056015 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33f9c6-b568-4e06-a85c-778e6f2a7543 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:25.205057 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1013/2022] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:08:24 2022] POST /identity/v3/projects => generated 375 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:25.210732 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33f9c6-b568-4e06-a85c-778e6f2a7543 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:25.218774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c89e7504-36c1-48c5-8235-bcb01838e3e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc1da801397c48a2831d83bc2f0ece0c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:25.219595 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c89e7504-36c1-48c5-8235-bcb01838e3e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc1da801397c48a2831d83bc2f0ece0c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:25.243000 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c89e7504-36c1-48c5-8235-bcb01838e3e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc1da801397c48a2831d83bc2f0ece0c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:25.254152 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c89e7504-36c1-48c5-8235-bcb01838e3e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc1da801397c48a2831d83bc2f0ece0c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:25.304422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33f9c6-b568-4e06-a85c-778e6f2a7543 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:25.312786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391ea2e-df31-49eb-aab8-ba3d0f8be849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:25.315031 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391ea2e-df31-49eb-aab8-ba3d0f8be849 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:25.316218 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391ea2e-df31-49eb-aab8-ba3d0f8be849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:25.316540 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391ea2e-df31-49eb-aab8-ba3d0f8be849 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:25.317263 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f391ea2e-df31-49eb-aab8-ba3d0f8be849 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:25.319575 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c89e7504-36c1-48c5-8235-bcb01838e3e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc1da801397c48a2831d83bc2f0ece0c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:25.321438 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f391ea2e-df31-49eb-aab8-ba3d0f8be849 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:25.329280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1010/2023] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:08:24 2022] DELETE /identity/v3/users/dc1da801397c48a2831d83bc2f0ece0c => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:25.330249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e7504-36c1-48c5-8235-bcb01838e3e3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:25.342042 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c89e7504-36c1-48c5-8235-bcb01838e3e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:25.417645 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1014/2024] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:08:25 2022] POST /identity/v3/users => generated 288 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:25.424112 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391ea2e-df31-49eb-aab8-ba3d0f8be849 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:25.424573 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f391ea2e-df31-49eb-aab8-ba3d0f8be849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:25.437411 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89e7504-36c1-48c5-8235-bcb01838e3e3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:25.439887 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391ea2e-df31-49eb-aab8-ba3d0f8be849 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:25.443038 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f747f7c-2e35-42d2-9952-adf850c0034c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:25.444004 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f747f7c-2e35-42d2-9952-adf850c0034c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:25.447040 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f747f7c-2e35-42d2-9952-adf850c0034c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:25.447040 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f747f7c-2e35-42d2-9952-adf850c0034c admin admin] PATH_INFO: `/v3/projects/eff258d9a57b49c1abd0dd440f79fcd6` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:25.447040 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f747f7c-2e35-42d2-9952-adf850c0034c admin admin] RBAC: Authorizing `identity:delete_project(project_id=eff258d9a57b49c1abd0dd440f79fcd6)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:25.448114 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b7961-c51c-46f9-9f45-f2097d7a0c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:25.448335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f747f7c-2e35-42d2-9952-adf850c0034c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:25.449230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b7961-c51c-46f9-9f45-f2097d7a0c3d admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:25.449722 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b7961-c51c-46f9-9f45-f2097d7a0c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:25.455614 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b7961-c51c-46f9-9f45-f2097d7a0c3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:25.456957 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59b7961-c51c-46f9-9f45-f2097d7a0c3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:25.458880 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59b7961-c51c-46f9-9f45-f2097d7a0c3d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:25.463339 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:25.477201 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1015/2025] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:08:25.480323 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b7961-c51c-46f9-9f45-f2097d7a0c3d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:25.493864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b7961-c51c-46f9-9f45-f2097d7a0c3d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:25.503260 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa06a9d-73b3-4dc9-af50-fdd16accc4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:25.507096 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa06a9d-73b3-4dc9-af50-fdd16accc4e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:25.507096 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa06a9d-73b3-4dc9-af50-fdd16accc4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:25.507096 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa06a9d-73b3-4dc9-af50-fdd16accc4e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:25.507096 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa06a9d-73b3-4dc9-af50-fdd16accc4e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:25.507476 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa06a9d-73b3-4dc9-af50-fdd16accc4e5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:25.519902 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1016/2026] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:25.524461 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa06a9d-73b3-4dc9-af50-fdd16accc4e5 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:25.535907 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa06a9d-73b3-4dc9-af50-fdd16accc4e5 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:25.541390 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a922a-dacd-4d48-a07d-d42da9002869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:25.542294 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a922a-dacd-4d48-a07d-d42da9002869 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:25.542582 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a922a-dacd-4d48-a07d-d42da9002869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:25.542863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a922a-dacd-4d48-a07d-d42da9002869 admin admin] PATH_INFO: `/v3/projects/506ce043e11e4b39b4115e82f56d184f/users/3e5dd9b04dcb4f2f86f1b8be00401ee3/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:25.572654 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a922a-dacd-4d48-a07d-d42da9002869 admin admin] RBAC: Authorizing `identity:create_grant(project_id=506ce043e11e4b39b4115e82f56d184f, user_id=3e5dd9b04dcb4f2f86f1b8be00401ee3, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:25.575490 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a922a-dacd-4d48-a07d-d42da9002869 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:25.639423 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:25.668794 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1017/2027] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:08:25 2022] PUT /identity/v3/projects/506ce043e11e4b39b4115e82f56d184f/users/3e5dd9b04dcb4f2f86f1b8be00401ee3/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:08:25.679725 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6f747f7c-2e35-42d2-9952-adf850c0034c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eff258d9a57b49c1abd0dd440f79fcd6', 'request_id': 'req-6f747f7c-2e35-42d2-9952-adf850c0034c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:25.680787 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1f26b-2ad7-4b36-91cc-4f7c91dc5249 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:25.681152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1f26b-2ad7-4b36-91cc-4f7c91dc5249 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:25.681636 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1f26b-2ad7-4b36-91cc-4f7c91dc5249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:25.686561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6f747f7c-2e35-42d2-9952-adf850c0034c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eff258d9a57b49c1abd0dd440f79fcd6', 'request_id': 'req-6f747f7c-2e35-42d2-9952-adf850c0034c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:25.692578 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6f747f7c-2e35-42d2-9952-adf850c0034c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eff258d9a57b49c1abd0dd440f79fcd6', 'request_id': 'req-6f747f7c-2e35-42d2-9952-adf850c0034c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:25.715008 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c7d1f26b-2ad7-4b36-91cc-4f7c91dc5249 None None] MFA Rules not processed for user `3e5dd9b04dcb4f2f86f1b8be00401ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:25.716250 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c7d1f26b-2ad7-4b36-91cc-4f7c91dc5249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:25.791917 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1018/2028] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:08:25.793179 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6f747f7c-2e35-42d2-9952-adf850c0034c admin admin] The token cache is being invalidate because project eff258d9a57b49c1abd0dd440f79fcd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:08:25.795540 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6f747f7c-2e35-42d2-9952-adf850c0034c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:25.796233 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6f747f7c-2e35-42d2-9952-adf850c0034c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:25.797015 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1f26b-2ad7-4b36-91cc-4f7c91dc5249 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:25.802875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1011/2029] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:08:25 2022] DELETE /identity/v3/projects/eff258d9a57b49c1abd0dd440f79fcd6 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:25.807324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d5b88-a8a1-4e48-8484-aaff52e381ab None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:25.807609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d5b88-a8a1-4e48-8484-aaff52e381ab None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:25.807888 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d5b88-a8a1-4e48-8484-aaff52e381ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:25.829039 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-fd9d5b88-a8a1-4e48-8484-aaff52e381ab None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:25.830870 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fd9d5b88-a8a1-4e48-8484-aaff52e381ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:25.860830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1f26b-2ad7-4b36-91cc-4f7c91dc5249 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:25.865853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cba4b2-3290-4e17-a8d1-98dadcf67cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:25.869996 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cba4b2-3290-4e17-a8d1-98dadcf67cce admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:25.869996 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cba4b2-3290-4e17-a8d1-98dadcf67cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:25.869996 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cba4b2-3290-4e17-a8d1-98dadcf67cce admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:25.869996 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cba4b2-3290-4e17-a8d1-98dadcf67cce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:25.869996 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cba4b2-3290-4e17-a8d1-98dadcf67cce admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:25.894695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1012/2030] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:25.899530 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d5b88-a8a1-4e48-8484-aaff52e381ab None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:25.903219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fd9d5b88-a8a1-4e48-8484-aaff52e381ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:25.916537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d5b88-a8a1-4e48-8484-aaff52e381ab None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:25.923419 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0474e292-679c-4bc2-840d-f5273c1ffc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:25.925028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474e292-679c-4bc2-840d-f5273c1ffc91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:25.925028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474e292-679c-4bc2-840d-f5273c1ffc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:25.925028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474e292-679c-4bc2-840d-f5273c1ffc91 admin admin] PATH_INFO: `/v3/projects/eed10daaed984c74adcd681a372b4ce6` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:25.927159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0474e292-679c-4bc2-840d-f5273c1ffc91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eed10daaed984c74adcd681a372b4ce6)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:25.928506 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0474e292-679c-4bc2-840d-f5273c1ffc91 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:25.987786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1019/2031] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:08:25 2022] POST /identity/v3/projects => generated 375 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:25.989232 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d47778-d3ea-4d1f-b2da-0fcba22fb27f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:25.991951 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d47778-d3ea-4d1f-b2da-0fcba22fb27f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:25.992253 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d47778-d3ea-4d1f-b2da-0fcba22fb27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:26.019575 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-91d47778-d3ea-4d1f-b2da-0fcba22fb27f None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:26.020960 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-91d47778-d3ea-4d1f-b2da-0fcba22fb27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:26.047208 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:26.107880 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0474e292-679c-4bc2-840d-f5273c1ffc91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eed10daaed984c74adcd681a372b4ce6', 'request_id': 'req-0474e292-679c-4bc2-840d-f5273c1ffc91'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:26.118475 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0474e292-679c-4bc2-840d-f5273c1ffc91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eed10daaed984c74adcd681a372b4ce6', 'request_id': 'req-0474e292-679c-4bc2-840d-f5273c1ffc91'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:26.124603 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0474e292-679c-4bc2-840d-f5273c1ffc91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eed10daaed984c74adcd681a372b4ce6', 'request_id': 'req-0474e292-679c-4bc2-840d-f5273c1ffc91'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:26.173916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1020/2032] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:26.178249 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d47778-d3ea-4d1f-b2da-0fcba22fb27f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:26.179657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-91d47778-d3ea-4d1f-b2da-0fcba22fb27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:26.192941 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d47778-d3ea-4d1f-b2da-0fcba22fb27f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:26.219309 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0474e292-679c-4bc2-840d-f5273c1ffc91 admin admin] The token cache is being invalidate because project eed10daaed984c74adcd681a372b4ce6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:08:26.219852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0474e292-679c-4bc2-840d-f5273c1ffc91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:26.221217 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0474e292-679c-4bc2-840d-f5273c1ffc91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:26.224117 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1013/2033] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:08:25 2022] DELETE /identity/v3/projects/eed10daaed984c74adcd681a372b4ce6 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:26.224117 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0474e292-679c-4bc2-840d-f5273c1ffc91 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:26.228500 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0474e292-679c-4bc2-840d-f5273c1ffc91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:26.273323 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86140a6c-b9c4-4557-a3e0-0a11a6f5b62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:26.274012 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86140a6c-b9c4-4557-a3e0-0a11a6f5b62f admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:26.274784 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86140a6c-b9c4-4557-a3e0-0a11a6f5b62f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:26.274784 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86140a6c-b9c4-4557-a3e0-0a11a6f5b62f admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:26.275307 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86140a6c-b9c4-4557-a3e0-0a11a6f5b62f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:26.277511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86140a6c-b9c4-4557-a3e0-0a11a6f5b62f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:26.306721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0474e292-679c-4bc2-840d-f5273c1ffc91 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:26.311858 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c2ccbd-812d-46c7-893e-ea4a4b0f793f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:26.312668 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2ccbd-812d-46c7-893e-ea4a4b0f793f admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:26.316384 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2ccbd-812d-46c7-893e-ea4a4b0f793f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:26.316384 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c2ccbd-812d-46c7-893e-ea4a4b0f793f admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:26.316384 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2ccbd-812d-46c7-893e-ea4a4b0f793f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:08:26.316384 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2ccbd-812d-46c7-893e-ea4a4b0f793f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:26.316765 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c2ccbd-812d-46c7-893e-ea4a4b0f793f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:26.330749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1014/2034] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:08:26.332089 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f5389-9f96-42d6-a018-d5d35d6b96df None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:26.332406 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f5389-9f96-42d6-a018-d5d35d6b96df None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:26.332689 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f5389-9f96-42d6-a018-d5d35d6b96df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:26.350572 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-bf8f5389-9f96-42d6-a018-d5d35d6b96df None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:26.351847 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-bf8f5389-9f96-42d6-a018-d5d35d6b96df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:26.359739 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1015/2035] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:26.360497 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f5389-9f96-42d6-a018-d5d35d6b96df None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:26.369826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f5389-9f96-42d6-a018-d5d35d6b96df None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:26.372716 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1021/2036] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:08:26 2022] POST /identity/v3/users => generated 288 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:26.372716 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16082908-49ce-47eb-9298-d08c6dbdb419 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:26.373024 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16082908-49ce-47eb-9298-d08c6dbdb419 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:26.373558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16082908-49ce-47eb-9298-d08c6dbdb419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:26.379490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e54a0c-a46f-4537-9098-931f9099834b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:26.380335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e54a0c-a46f-4537-9098-931f9099834b admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:26.380577 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e54a0c-a46f-4537-9098-931f9099834b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:26.380855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e54a0c-a46f-4537-9098-931f9099834b admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:26.381562 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e54a0c-a46f-4537-9098-931f9099834b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:26.383491 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e54a0c-a46f-4537-9098-931f9099834b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:26.395234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-16082908-49ce-47eb-9298-d08c6dbdb419 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:26.396483 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-16082908-49ce-47eb-9298-d08c6dbdb419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:26.404778 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1022/2037] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:26.405595 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16082908-49ce-47eb-9298-d08c6dbdb419 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:26.409561 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-16082908-49ce-47eb-9298-d08c6dbdb419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:26.420981 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16082908-49ce-47eb-9298-d08c6dbdb419 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:26.430978 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1033c123-0c1c-4349-a814-c6b242b95af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:26.430978 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1033c123-0c1c-4349-a814-c6b242b95af7 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:26.430978 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1033c123-0c1c-4349-a814-c6b242b95af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:26.430978 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1033c123-0c1c-4349-a814-c6b242b95af7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:26.431505 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1033c123-0c1c-4349-a814-c6b242b95af7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:26.431505 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1033c123-0c1c-4349-a814-c6b242b95af7 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:26.439826 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1023/2038] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:26.440636 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1033c123-0c1c-4349-a814-c6b242b95af7 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:26.450200 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1033c123-0c1c-4349-a814-c6b242b95af7 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:26.455473 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27b402-bd4c-41ea-9cf1-798eebe62fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:26.456100 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27b402-bd4c-41ea-9cf1-798eebe62fcb admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:26.456404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27b402-bd4c-41ea-9cf1-798eebe62fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:26.456678 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27b402-bd4c-41ea-9cf1-798eebe62fcb admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:26.457388 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27b402-bd4c-41ea-9cf1-798eebe62fcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:08:26.457690 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27b402-bd4c-41ea-9cf1-798eebe62fcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:26.459317 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27b402-bd4c-41ea-9cf1-798eebe62fcb admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:26.471103 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1024/2039] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:08:26.471943 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27b402-bd4c-41ea-9cf1-798eebe62fcb admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:26.481432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27b402-bd4c-41ea-9cf1-798eebe62fcb admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:26.486222 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afca6ef-f6ca-49bc-911a-bb5ce7bde77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:26.486896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afca6ef-f6ca-49bc-911a-bb5ce7bde77b admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:26.487175 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afca6ef-f6ca-49bc-911a-bb5ce7bde77b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:26.487492 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afca6ef-f6ca-49bc-911a-bb5ce7bde77b admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:26.488259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afca6ef-f6ca-49bc-911a-bb5ce7bde77b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:26.494312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afca6ef-f6ca-49bc-911a-bb5ce7bde77b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:26.494424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1016/2040] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:08:26 2022] POST /identity/v3/projects => generated 383 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:26.495129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e54a0c-a46f-4537-9098-931f9099834b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:26.503229 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1025/2041] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:26.504037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afca6ef-f6ca-49bc-911a-bb5ce7bde77b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:26.513916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afca6ef-f6ca-49bc-911a-bb5ce7bde77b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:26.549696 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:26.560480 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e54a0c-a46f-4537-9098-931f9099834b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:26.566112 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14aa868-b454-48cb-91c9-427992c8f26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:26.566772 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14aa868-b454-48cb-91c9-427992c8f26d admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:26.567058 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14aa868-b454-48cb-91c9-427992c8f26d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:26.567338 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14aa868-b454-48cb-91c9-427992c8f26d admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:26.568034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14aa868-b454-48cb-91c9-427992c8f26d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:26.570137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14aa868-b454-48cb-91c9-427992c8f26d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:26.570249 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fecf47-7645-44ed-9327-dc9e1e36e0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:26.570880 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fecf47-7645-44ed-9327-dc9e1e36e0bd admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:26.571163 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fecf47-7645-44ed-9327-dc9e1e36e0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:26.571442 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fecf47-7645-44ed-9327-dc9e1e36e0bd admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:26.572424 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fecf47-7645-44ed-9327-dc9e1e36e0bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:26.574718 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fecf47-7645-44ed-9327-dc9e1e36e0bd admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:26.725046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1026/2042] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:08:26 2022] POST /identity/v3/users => generated 292 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:26.725773 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fecf47-7645-44ed-9327-dc9e1e36e0bd admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:26.735259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fecf47-7645-44ed-9327-dc9e1e36e0bd admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:26.741910 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b5266-cf4e-4fe3-b8f8-12c059aa31e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:26.743071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b5266-cf4e-4fe3-b8f8-12c059aa31e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:26.743355 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b5266-cf4e-4fe3-b8f8-12c059aa31e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:26.743635 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b5266-cf4e-4fe3-b8f8-12c059aa31e6 admin admin] PATH_INFO: `/v3/projects/b44fa59ba673426f9990fea70f71c8e7/users/af7015bb2cb3499da7b73b95db457c89/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:26.759332 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1017/2043] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:08:26 2022] POST /identity/v3/projects => generated 375 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:26.759332 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14aa868-b454-48cb-91c9-427992c8f26d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:26.763366 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b5266-cf4e-4fe3-b8f8-12c059aa31e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b44fa59ba673426f9990fea70f71c8e7, user_id=af7015bb2cb3499da7b73b95db457c89, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:26.766179 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b5266-cf4e-4fe3-b8f8-12c059aa31e6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:26.819331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14aa868-b454-48cb-91c9-427992c8f26d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:26.824333 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966058c9-270f-425b-b73f-22621e96f691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:26.824986 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966058c9-270f-425b-b73f-22621e96f691 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:26.825266 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966058c9-270f-425b-b73f-22621e96f691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:26.825574 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966058c9-270f-425b-b73f-22621e96f691 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:26.826319 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966058c9-270f-425b-b73f-22621e96f691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:26.827906 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966058c9-270f-425b-b73f-22621e96f691 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:26.837468 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1018/2044] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:26.838233 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966058c9-270f-425b-b73f-22621e96f691 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:26.847577 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966058c9-270f-425b-b73f-22621e96f691 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:26.852525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e5e64-020e-4295-bfd2-8d4974d4f182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:26.853157 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e5e64-020e-4295-bfd2-8d4974d4f182 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:26.853433 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e5e64-020e-4295-bfd2-8d4974d4f182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:26.853702 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e5e64-020e-4295-bfd2-8d4974d4f182 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:26.854401 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e5e64-020e-4295-bfd2-8d4974d4f182 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:26.856580 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e5e64-020e-4295-bfd2-8d4974d4f182 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:26.868954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1027/2045] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:08:26 2022] PUT /identity/v3/projects/b44fa59ba673426f9990fea70f71c8e7/users/af7015bb2cb3499da7b73b95db457c89/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:26.870092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b5266-cf4e-4fe3-b8f8-12c059aa31e6 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:26.934792 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b5266-cf4e-4fe3-b8f8-12c059aa31e6 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:26.939854 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45354e-8189-466b-b72c-37dbda025a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:26.940609 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45354e-8189-466b-b72c-37dbda025a11 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:26.940897 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45354e-8189-466b-b72c-37dbda025a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:26.941180 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45354e-8189-466b-b72c-37dbda025a11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:26.941912 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45354e-8189-466b-b72c-37dbda025a11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:26.943661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45354e-8189-466b-b72c-37dbda025a11 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:26.954874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1028/2046] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:26.956351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5b740-3055-49c8-8404-f000993692c2 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:26.956645 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5b740-3055-49c8-8404-f000993692c2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:26.956923 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda5b740-3055-49c8-8404-f000993692c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:26.983230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-dda5b740-3055-49c8-8404-f000993692c2 None None] MFA Rules not processed for user `af7015bb2cb3499da7b73b95db457c89`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:26.984474 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-dda5b740-3055-49c8-8404-f000993692c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:27.006046 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1019/2047] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:08:26 2022] POST /identity/v3/users => generated 288 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:27.006916 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e5e64-020e-4295-bfd2-8d4974d4f182 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:27.018563 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e5e64-020e-4295-bfd2-8d4974d4f182 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:27.023515 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee30de-45e7-4282-a15a-b895aa85a900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:27.024513 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee30de-45e7-4282-a15a-b895aa85a900 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:27.027510 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee30de-45e7-4282-a15a-b895aa85a900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:27.027510 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee30de-45e7-4282-a15a-b895aa85a900 admin admin] PATH_INFO: `/v3/projects/cbb4cea2394847a4994be78222d4aa83/users/6f53906c1b1b4de4b2c39a4c1551f68d/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:27.042185 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee30de-45e7-4282-a15a-b895aa85a900 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbb4cea2394847a4994be78222d4aa83, user_id=6f53906c1b1b4de4b2c39a4c1551f68d, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:27.045170 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee30de-45e7-4282-a15a-b895aa85a900 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:27.059842 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1029/2048] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:27.059842 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda5b740-3055-49c8-8404-f000993692c2 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:27.071387 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda5b740-3055-49c8-8404-f000993692c2 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:27.076712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2dc11-0955-4a87-9b0c-8e549664aac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:27.077391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2dc11-0955-4a87-9b0c-8e549664aac6 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:27.077680 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2dc11-0955-4a87-9b0c-8e549664aac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:27.077989 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2dc11-0955-4a87-9b0c-8e549664aac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:27.078661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2dc11-0955-4a87-9b0c-8e549664aac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:27.080265 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2dc11-0955-4a87-9b0c-8e549664aac6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:27.090894 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1030/2049] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:27.091699 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2dc11-0955-4a87-9b0c-8e549664aac6 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:27.104769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2dc11-0955-4a87-9b0c-8e549664aac6 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:27.110193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf57fb-78d1-4574-a077-f8f0ca6bcb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:27.111437 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf57fb-78d1-4574-a077-f8f0ca6bcb84 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:27.111722 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf57fb-78d1-4574-a077-f8f0ca6bcb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:27.113977 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf57fb-78d1-4574-a077-f8f0ca6bcb84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:27.113977 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf57fb-78d1-4574-a077-f8f0ca6bcb84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:27.114712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bf57fb-78d1-4574-a077-f8f0ca6bcb84 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:27.148765 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1020/2050] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:08:27 2022] PUT /identity/v3/projects/cbb4cea2394847a4994be78222d4aa83/users/6f53906c1b1b4de4b2c39a4c1551f68d/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:27.152137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee30de-45e7-4282-a15a-b895aa85a900 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:27.213435 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee30de-45e7-4282-a15a-b895aa85a900 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:27.218133 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29343808-6e32-4871-a58e-c6f412a5ea4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:27.218792 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29343808-6e32-4871-a58e-c6f412a5ea4b admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:27.219073 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29343808-6e32-4871-a58e-c6f412a5ea4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:27.219421 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29343808-6e32-4871-a58e-c6f412a5ea4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:27.220618 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29343808-6e32-4871-a58e-c6f412a5ea4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:27.222202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29343808-6e32-4871-a58e-c6f412a5ea4b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:27.237914 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1021/2051] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:27.237914 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be711a76-5d2b-4074-bb06-d82d140e4b26 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:27.237914 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be711a76-5d2b-4074-bb06-d82d140e4b26 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:27.237914 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be711a76-5d2b-4074-bb06-d82d140e4b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:27.264220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1031/2052] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:08:27 2022] POST /identity/v3/projects => generated 375 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:27.269210 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf57fb-78d1-4574-a077-f8f0ca6bcb84 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:27.271345 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-be711a76-5d2b-4074-bb06-d82d140e4b26 None None] MFA Rules not processed for user `6f53906c1b1b4de4b2c39a4c1551f68d`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:27.273075 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-be711a76-5d2b-4074-bb06-d82d140e4b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:27.344520 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf57fb-78d1-4574-a077-f8f0ca6bcb84 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:27.351386 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1022/2053] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:27.351386 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be711a76-5d2b-4074-bb06-d82d140e4b26 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:27.352031 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16143e-3e63-427e-8684-11ba157ed036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:27.352031 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16143e-3e63-427e-8684-11ba157ed036 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:27.352031 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16143e-3e63-427e-8684-11ba157ed036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:27.352031 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16143e-3e63-427e-8684-11ba157ed036 admin admin] PATH_INFO: `/v3/projects/4f03ef8c22ad4b58b93c6665bb2f5bba/users/013d57b8957840a7bfc98ff17597b776/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:27.361076 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be711a76-5d2b-4074-bb06-d82d140e4b26 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:27.366084 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5994c-9180-4b0c-9753-5633044dc7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:27.366761 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5994c-9180-4b0c-9753-5633044dc7f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:27.368878 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5994c-9180-4b0c-9753-5633044dc7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:27.368878 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5994c-9180-4b0c-9753-5633044dc7f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:27.368878 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5994c-9180-4b0c-9753-5633044dc7f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:27.369157 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16143e-3e63-427e-8684-11ba157ed036 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f03ef8c22ad4b58b93c6665bb2f5bba, user_id=013d57b8957840a7bfc98ff17597b776, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:27.370068 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5994c-9180-4b0c-9753-5633044dc7f8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:27.371488 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16143e-3e63-427e-8684-11ba157ed036 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:27.410619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:27.463243 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1032/2054] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:08:27 2022] PUT /identity/v3/projects/4f03ef8c22ad4b58b93c6665bb2f5bba/users/013d57b8957840a7bfc98ff17597b776/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:27.473331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16143e-3e63-427e-8684-11ba157ed036 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:27.479086 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:27.483583 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1023/2055] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:08:27 2022] POST /identity/v3/users => generated 294 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:27.485563 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b43fd-f326-417c-b7cf-fb87f77e2597 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:27.486068 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b43fd-f326-417c-b7cf-fb87f77e2597 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:27.486959 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b43fd-f326-417c-b7cf-fb87f77e2597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:27.517915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-654b43fd-f326-417c-b7cf-fb87f77e2597 None None] MFA Rules not processed for user `013d57b8957840a7bfc98ff17597b776`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:27.519207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-654b43fd-f326-417c-b7cf-fb87f77e2597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:27.560537 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16143e-3e63-427e-8684-11ba157ed036 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:27.568334 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af780c-0d8b-4be0-8168-34d8027a7227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:27.569475 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af780c-0d8b-4be0-8168-34d8027a7227 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:27.570256 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af780c-0d8b-4be0-8168-34d8027a7227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:27.570392 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af780c-0d8b-4be0-8168-34d8027a7227 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:27.574302 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af780c-0d8b-4be0-8168-34d8027a7227 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:27.575633 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af780c-0d8b-4be0-8168-34d8027a7227 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:27.607603 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1024/2056] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:27.607603 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b43fd-f326-417c-b7cf-fb87f77e2597 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:27.623498 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b43fd-f326-417c-b7cf-fb87f77e2597 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:27.633247 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f14137-ddfa-46c7-8564-0bbb4e40604b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:27.633920 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f14137-ddfa-46c7-8564-0bbb4e40604b admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:27.634974 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f14137-ddfa-46c7-8564-0bbb4e40604b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:27.634974 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f14137-ddfa-46c7-8564-0bbb4e40604b admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:27.635342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f14137-ddfa-46c7-8564-0bbb4e40604b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:27.637290 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f14137-ddfa-46c7-8564-0bbb4e40604b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:27.650526 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1025/2057] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:27.652487 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f14137-ddfa-46c7-8564-0bbb4e40604b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:27.653345 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-36f14137-ddfa-46c7-8564-0bbb4e40604b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:27.677518 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1033/2058] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:08:27 2022] POST /identity/v3/projects => generated 385 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:27.677787 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af780c-0d8b-4be0-8168-34d8027a7227 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:27.739101 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f14137-ddfa-46c7-8564-0bbb4e40604b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:27.756111 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af780c-0d8b-4be0-8168-34d8027a7227 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:27.761543 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c10c25-69fd-44ca-a9a3-6c3757e809b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:27.762541 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c10c25-69fd-44ca-a9a3-6c3757e809b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:27.762902 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c10c25-69fd-44ca-a9a3-6c3757e809b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:27.763251 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c10c25-69fd-44ca-a9a3-6c3757e809b0 admin admin] PATH_INFO: `/v3/projects/af1527b3e173427e893fff95b811fc30/users/55007a692d954c0e991a3770a7f6c0a1/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:27.783799 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c10c25-69fd-44ca-a9a3-6c3757e809b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af1527b3e173427e893fff95b811fc30, user_id=55007a692d954c0e991a3770a7f6c0a1, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:27.787078 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c10c25-69fd-44ca-a9a3-6c3757e809b0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:27.795612 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e5177-7477-440e-9c93-b15831ffaed3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd72f952df34ad8853fe84673a41f45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:27.796286 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e5177-7477-440e-9c93-b15831ffaed3 service glance] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:27.796597 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e5177-7477-440e-9c93-b15831ffaed3 service glance] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:27.806537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e5177-7477-440e-9c93-b15831ffaed3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:27.870053 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e5177-7477-440e-9c93-b15831ffaed3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:27.871318 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e5177-7477-440e-9c93-b15831ffaed3 service glance] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:27.888879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1026/2060] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:08:27.889082 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1034/2060] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:08:27 2022] PUT /identity/v3/projects/af1527b3e173427e893fff95b811fc30/users/55007a692d954c0e991a3770a7f6c0a1/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:27.889998 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c10c25-69fd-44ca-a9a3-6c3757e809b0 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:27.890949 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e5177-7477-440e-9c93-b15831ffaed3 service glance] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:27.908576 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e5177-7477-440e-9c93-b15831ffaed3 service glance] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:27.990577 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b9bd0-a408-4de6-a3aa-f03f7aecb061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:27.991287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b9bd0-a408-4de6-a3aa-f03f7aecb061 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:27.991723 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b9bd0-a408-4de6-a3aa-f03f7aecb061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:27.992017 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b9bd0-a408-4de6-a3aa-f03f7aecb061 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:27.992859 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b9bd0-a408-4de6-a3aa-f03f7aecb061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:27.994552 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b9bd0-a408-4de6-a3aa-f03f7aecb061 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:28.013269 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1027/2061] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:28.015405 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b9bd0-a408-4de6-a3aa-f03f7aecb061 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:28.032094 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b9bd0-a408-4de6-a3aa-f03f7aecb061 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:28.034230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c10c25-69fd-44ca-a9a3-6c3757e809b0 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:28.037713 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e313f5bd-a7b1-4f79-a387-2a012eeaed51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:28.039388 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313f5bd-a7b1-4f79-a387-2a012eeaed51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:28.039766 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313f5bd-a7b1-4f79-a387-2a012eeaed51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:28.040054 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313f5bd-a7b1-4f79-a387-2a012eeaed51 admin admin] PATH_INFO: `/v3/projects/af1527b3e173427e893fff95b811fc30/users/55007a692d954c0e991a3770a7f6c0a1/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:28.041181 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ee931-4147-4597-bac9-c862259cd2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:28.042504 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ee931-4147-4597-bac9-c862259cd2e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:28.042849 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e313f5bd-a7b1-4f79-a387-2a012eeaed51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af1527b3e173427e893fff95b811fc30, user_id=55007a692d954c0e991a3770a7f6c0a1, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:28.042962 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ee931-4147-4597-bac9-c862259cd2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:28.043731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ee931-4147-4597-bac9-c862259cd2e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:28.045117 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ee931-4147-4597-bac9-c862259cd2e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:28.045980 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e313f5bd-a7b1-4f79-a387-2a012eeaed51 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:28.048927 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ee931-4147-4597-bac9-c862259cd2e9 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:28.068358 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1028/2062] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:08:28 2022] PUT /identity/v3/projects/af1527b3e173427e893fff95b811fc30/users/55007a692d954c0e991a3770a7f6c0a1/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:28.072491 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d1db1-741a-4041-9ff1-bfdc0b0878b2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:28.072807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d1db1-741a-4041-9ff1-bfdc0b0878b2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:28.073330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8d1db1-741a-4041-9ff1-bfdc0b0878b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:28.102849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:28.104236 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-aa8d1db1-741a-4041-9ff1-bfdc0b0878b2 None None] MFA Rules not processed for user `55007a692d954c0e991a3770a7f6c0a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:28.105609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-aa8d1db1-741a-4041-9ff1-bfdc0b0878b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:28.155908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1035/2063] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:08:27 2022] POST /identity/v3/users => generated 299 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:28.160066 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ee931-4147-4597-bac9-c862259cd2e9 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:28.171114 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ee931-4147-4597-bac9-c862259cd2e9 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:28.176693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:28.190526 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1029/2064] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:28 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:28.203304 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d1db1-741a-4041-9ff1-bfdc0b0878b2 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:28.205896 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-aa8d1db1-741a-4041-9ff1-bfdc0b0878b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:28.266010 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f31dd-5888-450c-8a82-4c55d9c95965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:28.266669 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f31dd-5888-450c-8a82-4c55d9c95965 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:28.267285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f31dd-5888-450c-8a82-4c55d9c95965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:28.267594 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f31dd-5888-450c-8a82-4c55d9c95965 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:28.268808 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f31dd-5888-450c-8a82-4c55d9c95965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:28.271351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f31dd-5888-450c-8a82-4c55d9c95965 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:28.282081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8d1db1-741a-4041-9ff1-bfdc0b0878b2 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:28.284476 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1036/2065] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:28.288335 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f31dd-5888-450c-8a82-4c55d9c95965 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:28.292154 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54cfd9-5875-4c95-9f55-3bd99bd7033e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:28.293092 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54cfd9-5875-4c95-9f55-3bd99bd7033e service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:28.293377 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54cfd9-5875-4c95-9f55-3bd99bd7033e service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:28.293713 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a54cfd9-5875-4c95-9f55-3bd99bd7033e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:28.296521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5a54cfd9-5875-4c95-9f55-3bd99bd7033e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:28.301083 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f31dd-5888-450c-8a82-4c55d9c95965 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:28.318795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbbdb8-0606-4c3a-a395-86cc15072978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:28.319788 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbbdb8-0606-4c3a-a395-86cc15072978 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:28.320072 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbbdb8-0606-4c3a-a395-86cc15072978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:28.320644 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbbdb8-0606-4c3a-a395-86cc15072978 admin admin] PATH_INFO: `/v3/projects/161fdbb8c7ae4ab89f4df22482076b9e/users/243793d3d042416ab46109cf4a2d5088/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:28.338005 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cbbdb8-0606-4c3a-a395-86cc15072978 admin admin] RBAC: Authorizing `identity:create_grant(project_id=161fdbb8c7ae4ab89f4df22482076b9e, user_id=243793d3d042416ab46109cf4a2d5088, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:28.341971 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cbbdb8-0606-4c3a-a395-86cc15072978 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:28.394179 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a54cfd9-5875-4c95-9f55-3bd99bd7033e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:28.394179 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a54cfd9-5875-4c95-9f55-3bd99bd7033e service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:28.399924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1030/2066] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 20:08:28.417902 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1037/2067] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:08:28 2022] PUT /identity/v3/projects/161fdbb8c7ae4ab89f4df22482076b9e/users/243793d3d042416ab46109cf4a2d5088/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:28.422360 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:28.425685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54cfd9-5875-4c95-9f55-3bd99bd7033e service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:28.449252 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a54cfd9-5875-4c95-9f55-3bd99bd7033e service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:28.484783 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:28.522654 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809abd2a-1786-4aad-8653-b605b5c357b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:28.522654 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809abd2a-1786-4aad-8653-b605b5c357b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:28.522654 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809abd2a-1786-4aad-8653-b605b5c357b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:28.522654 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809abd2a-1786-4aad-8653-b605b5c357b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:28.523350 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809abd2a-1786-4aad-8653-b605b5c357b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:28.523350 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809abd2a-1786-4aad-8653-b605b5c357b3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:28.533629 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1031/2068] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:28.537918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbbdb8-0606-4c3a-a395-86cc15072978 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:28.548962 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbbdb8-0606-4c3a-a395-86cc15072978 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:28.554114 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e9c32-4348-4fc5-947c-7d822e8183b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:28.554994 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e9c32-4348-4fc5-947c-7d822e8183b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:28.555273 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e9c32-4348-4fc5-947c-7d822e8183b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:28.561090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e9c32-4348-4fc5-947c-7d822e8183b2 admin admin] PATH_INFO: `/v3/projects/161fdbb8c7ae4ab89f4df22482076b9e/users/243793d3d042416ab46109cf4a2d5088/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:28.561090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e9c32-4348-4fc5-947c-7d822e8183b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=161fdbb8c7ae4ab89f4df22482076b9e, user_id=243793d3d042416ab46109cf4a2d5088, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:28.561090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e9c32-4348-4fc5-947c-7d822e8183b2 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:28.572091 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1038/2069] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:08:28 2022] PUT /identity/v3/projects/161fdbb8c7ae4ab89f4df22482076b9e/users/243793d3d042416ab46109cf4a2d5088/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:28.582350 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e381d-2589-4e7e-8f88-953a33433f5c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:28.582641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e381d-2589-4e7e-8f88-953a33433f5c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:28.582919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e381d-2589-4e7e-8f88-953a33433f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:28.618207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-420e381d-2589-4e7e-8f88-953a33433f5c None None] MFA Rules not processed for user `243793d3d042416ab46109cf4a2d5088`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:28.619437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-420e381d-2589-4e7e-8f88-953a33433f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:28.691857 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1032/2070] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:29.250258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e9c32-4348-4fc5-947c-7d822e8183b2 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:29.252479 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-299e9c32-4348-4fc5-947c-7d822e8183b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:29.318264 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e9c32-4348-4fc5-947c-7d822e8183b2 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:29.323155 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757bdf2-aa11-4bdf-bf42-c9c91e95bc7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:29.326279 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757bdf2-aa11-4bdf-bf42-c9c91e95bc7f service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:29.326279 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757bdf2-aa11-4bdf-bf42-c9c91e95bc7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:29.326279 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3757bdf2-aa11-4bdf-bf42-c9c91e95bc7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:29.403365 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3757bdf2-aa11-4bdf-bf42-c9c91e95bc7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:29.403365 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3757bdf2-aa11-4bdf-bf42-c9c91e95bc7f service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:29.417151 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:29.427316 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1039/2071] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:08:30.028578 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:30.831641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e381d-2589-4e7e-8f88-953a33433f5c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:30.840999 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e381d-2589-4e7e-8f88-953a33433f5c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:30.844579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29457e65-e734-4801-98f6-448f0f56e12e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:30.845774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29457e65-e734-4801-98f6-448f0f56e12e service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:30.846066 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29457e65-e734-4801-98f6-448f0f56e12e service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:30.846337 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29457e65-e734-4801-98f6-448f0f56e12e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:30.903137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29457e65-e734-4801-98f6-448f0f56e12e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:30.904507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29457e65-e734-4801-98f6-448f0f56e12e service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:30.915659 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1033/2072] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:08:30.925764 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:31.475534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:32.182607 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:32.730587 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:32.781628 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757bdf2-aa11-4bdf-bf42-c9c91e95bc7f service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:32.790893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3757bdf2-aa11-4bdf-bf42-c9c91e95bc7f service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:32.794505 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61079a-adbe-47f5-86b4-90f0c7376ca6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:32.797797 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61079a-adbe-47f5-86b4-90f0c7376ca6 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:32.797797 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61079a-adbe-47f5-86b4-90f0c7376ca6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:32.797797 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61079a-adbe-47f5-86b4-90f0c7376ca6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:32.874037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a61079a-adbe-47f5-86b4-90f0c7376ca6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:32.878065 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a61079a-adbe-47f5-86b4-90f0c7376ca6 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:32.887702 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1040/2073] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:32 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:08:32.948153 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:33.147158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:33.285045 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:33.598981 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:33.771048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:34.004993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:34.202270 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:34.245104 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:34.434055 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:34.841368 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29457e65-e734-4801-98f6-448f0f56e12e service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:34.852056 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29457e65-e734-4801-98f6-448f0f56e12e service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:34.857114 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c431077-702c-4491-88e3-a4fbd161c4ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:34.858024 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c431077-702c-4491-88e3-a4fbd161c4ac service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:34.858575 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c431077-702c-4491-88e3-a4fbd161c4ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:34.859048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c431077-702c-4491-88e3-a4fbd161c4ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:34.873774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c431077-702c-4491-88e3-a4fbd161c4ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:34.875720 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c431077-702c-4491-88e3-a4fbd161c4ac service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:34.899141 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1034/2074] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:08:34.902759 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:35.028096 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:35.743155 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:36.425096 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61079a-adbe-47f5-86b4-90f0c7376ca6 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:36.426418 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0a61079a-adbe-47f5-86b4-90f0c7376ca6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:36.504051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61079a-adbe-47f5-86b4-90f0c7376ca6 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:36.508651 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091ddd9-3aec-45e6-b27a-d013c51e1e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:36.509493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091ddd9-3aec-45e6-b27a-d013c51e1e08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:36.509773 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091ddd9-3aec-45e6-b27a-d013c51e1e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:36.510043 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091ddd9-3aec-45e6-b27a-d013c51e1e08 admin admin] PATH_INFO: `/v3/users/5645338e8194426eac77359c0cb233f5` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:36.511221 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091ddd9-3aec-45e6-b27a-d013c51e1e08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5645338e8194426eac77359c0cb233f5)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:36.513230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f091ddd9-3aec-45e6-b27a-d013c51e1e08 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:36.531263 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:36.790086 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f091ddd9-3aec-45e6-b27a-d013c51e1e08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5645338e8194426eac77359c0cb233f5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:36.790515 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f091ddd9-3aec-45e6-b27a-d013c51e1e08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5645338e8194426eac77359c0cb233f5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:36.890565 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f091ddd9-3aec-45e6-b27a-d013c51e1e08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5645338e8194426eac77359c0cb233f5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:36.891210 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f091ddd9-3aec-45e6-b27a-d013c51e1e08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5645338e8194426eac77359c0cb233f5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:36.904529 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f091ddd9-3aec-45e6-b27a-d013c51e1e08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5645338e8194426eac77359c0cb233f5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:36.917652 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1041/2075] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:08:36 2022] DELETE /identity/v3/users/5645338e8194426eac77359c0cb233f5 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:37.154934 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c431077-702c-4491-88e3-a4fbd161c4ac service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:37.166717 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c431077-702c-4491-88e3-a4fbd161c4ac service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:37.232903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73158f30-2b29-402e-b8ca-883b6eb74855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:37.233785 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73158f30-2b29-402e-b8ca-883b6eb74855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:37.234148 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73158f30-2b29-402e-b8ca-883b6eb74855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:37.234433 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73158f30-2b29-402e-b8ca-883b6eb74855 admin admin] PATH_INFO: `/v3/projects/32672f6e771b4d37986e48e0091b6877` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:37.236838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73158f30-2b29-402e-b8ca-883b6eb74855 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32672f6e771b4d37986e48e0091b6877)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:37.239033 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73158f30-2b29-402e-b8ca-883b6eb74855 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:37.382550 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-73158f30-2b29-402e-b8ca-883b6eb74855 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32672f6e771b4d37986e48e0091b6877', 'request_id': 'req-73158f30-2b29-402e-b8ca-883b6eb74855'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:37.388585 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-73158f30-2b29-402e-b8ca-883b6eb74855 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32672f6e771b4d37986e48e0091b6877', 'request_id': 'req-73158f30-2b29-402e-b8ca-883b6eb74855'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:37.394044 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-73158f30-2b29-402e-b8ca-883b6eb74855 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32672f6e771b4d37986e48e0091b6877', 'request_id': 'req-73158f30-2b29-402e-b8ca-883b6eb74855'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:37.508252 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-73158f30-2b29-402e-b8ca-883b6eb74855 admin admin] The token cache is being invalidate because project 32672f6e771b4d37986e48e0091b6877 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:08:37.508584 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-73158f30-2b29-402e-b8ca-883b6eb74855 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:37.509200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-73158f30-2b29-402e-b8ca-883b6eb74855 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:37.510750 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1035/2076] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:08:37 2022] DELETE /identity/v3/projects/32672f6e771b4d37986e48e0091b6877 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:37.517116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a325a-4410-41d8-a9ca-d7e1401d90b3 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:37.519407 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a325a-4410-41d8-a9ca-d7e1401d90b3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:37.519407 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a325a-4410-41d8-a9ca-d7e1401d90b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:37.535528 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-857a325a-4410-41d8-a9ca-d7e1401d90b3 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:37.536786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-857a325a-4410-41d8-a9ca-d7e1401d90b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:37.597874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1042/2077] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:37.606082 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01184ddf-4211-4755-b2d5-54ad5b00bd34 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:37.606530 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01184ddf-4211-4755-b2d5-54ad5b00bd34 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:37.606927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01184ddf-4211-4755-b2d5-54ad5b00bd34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:37.627770 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-01184ddf-4211-4755-b2d5-54ad5b00bd34 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:37.629035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-01184ddf-4211-4755-b2d5-54ad5b00bd34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:37.640072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1036/2078] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:37.641523 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a325a-4410-41d8-a9ca-d7e1401d90b3 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:37.651290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a325a-4410-41d8-a9ca-d7e1401d90b3 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:37.656121 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88278d-35c3-4e3f-b585-663b028d7714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:37.656804 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88278d-35c3-4e3f-b585-663b028d7714 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:37.657591 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88278d-35c3-4e3f-b585-663b028d7714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:37.657938 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88278d-35c3-4e3f-b585-663b028d7714 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:37.658712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f88278d-35c3-4e3f-b585-663b028d7714 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:08:37.659018 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f88278d-35c3-4e3f-b585-663b028d7714 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:37.660672 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f88278d-35c3-4e3f-b585-663b028d7714 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:37.672593 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1043/2079] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:08:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:08:37.677467 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01184ddf-4211-4755-b2d5-54ad5b00bd34 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:37.686585 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01184ddf-4211-4755-b2d5-54ad5b00bd34 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:37.692588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e6b55-eed6-4776-a1e8-8b34b2b29780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:37.695388 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e6b55-eed6-4776-a1e8-8b34b2b29780 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:37.695388 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e6b55-eed6-4776-a1e8-8b34b2b29780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:37.695388 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e6b55-eed6-4776-a1e8-8b34b2b29780 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:37.695660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e6b55-eed6-4776-a1e8-8b34b2b29780 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:37.697869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e6b55-eed6-4776-a1e8-8b34b2b29780 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:37.795684 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1037/2080] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:08:37 2022] POST /identity/v3/projects => generated 371 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:37.799967 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88278d-35c3-4e3f-b585-663b028d7714 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:37.810407 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88278d-35c3-4e3f-b585-663b028d7714 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:37.870418 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133dbf7d-1928-4518-977b-5371365b3d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:37.871493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133dbf7d-1928-4518-977b-5371365b3d23 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:37.872138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133dbf7d-1928-4518-977b-5371365b3d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:37.872703 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133dbf7d-1928-4518-977b-5371365b3d23 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:37.873868 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133dbf7d-1928-4518-977b-5371365b3d23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:37.877894 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133dbf7d-1928-4518-977b-5371365b3d23 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:37.945658 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e6b55-eed6-4776-a1e8-8b34b2b29780 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:37.947693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0c8e6b55-eed6-4776-a1e8-8b34b2b29780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:37.957960 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1044/2081] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:08:37 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:37.958926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133dbf7d-1928-4518-977b-5371365b3d23 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:37.968713 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e6b55-eed6-4776-a1e8-8b34b2b29780 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:37.972126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133dbf7d-1928-4518-977b-5371365b3d23 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:37.974070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ccdbb-1ea3-4128-a159-42fae3864003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:37.974959 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ccdbb-1ea3-4128-a159-42fae3864003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:37.975213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ccdbb-1ea3-4128-a159-42fae3864003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:37.975494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ccdbb-1ea3-4128-a159-42fae3864003 admin admin] PATH_INFO: `/v3/users/3e5dd9b04dcb4f2f86f1b8be00401ee3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:37.976708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ccdbb-1ea3-4128-a159-42fae3864003 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e5dd9b04dcb4f2f86f1b8be00401ee3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:37.976990 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd5b17-63cc-4efb-bbb4-62edb988fe28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:37.977652 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd5b17-63cc-4efb-bbb4-62edb988fe28 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:37.977966 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd5b17-63cc-4efb-bbb4-62edb988fe28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:37.978245 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd5b17-63cc-4efb-bbb4-62edb988fe28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:37.978975 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bd5b17-63cc-4efb-bbb4-62edb988fe28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:37.980750 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bd5b17-63cc-4efb-bbb4-62edb988fe28 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:37.981103 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ccdbb-1ea3-4128-a159-42fae3864003 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:37.987020 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:37.993801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1045/2082] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:38.000107 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd5b17-63cc-4efb-bbb4-62edb988fe28 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:38.010468 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd5b17-63cc-4efb-bbb4-62edb988fe28 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:38.015535 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87386ab-e60e-4d1b-8aa4-4b5a5096144f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:38.016065 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87386ab-e60e-4d1b-8aa4-4b5a5096144f admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:38.017239 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87386ab-e60e-4d1b-8aa4-4b5a5096144f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:38.017755 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87386ab-e60e-4d1b-8aa4-4b5a5096144f admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:38.019090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87386ab-e60e-4d1b-8aa4-4b5a5096144f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:38.032949 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87386ab-e60e-4d1b-8aa4-4b5a5096144f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:38.032949 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1046/2083] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:38.036597 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87386ab-e60e-4d1b-8aa4-4b5a5096144f admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:38.048940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87386ab-e60e-4d1b-8aa4-4b5a5096144f admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:38.053529 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737caa1e-db9a-4273-acee-8012f1025998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:38.054449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737caa1e-db9a-4273-acee-8012f1025998 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:38.054731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737caa1e-db9a-4273-acee-8012f1025998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:38.055002 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737caa1e-db9a-4273-acee-8012f1025998 admin admin] PATH_INFO: `/v3/projects/8c7b02690ff34b6fb25d25e755d06a97/users/734e54fa10cd4a12bc3df01de6fb604b/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:38.071142 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737caa1e-db9a-4273-acee-8012f1025998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c7b02690ff34b6fb25d25e755d06a97, user_id=734e54fa10cd4a12bc3df01de6fb604b, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:38.075404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737caa1e-db9a-4273-acee-8012f1025998 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:38.145183 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:38.152773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e96ccdbb-1ea3-4128-a159-42fae3864003 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e5dd9b04dcb4f2f86f1b8be00401ee3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:38.153612 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e96ccdbb-1ea3-4128-a159-42fae3864003 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e5dd9b04dcb4f2f86f1b8be00401ee3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:38.158037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1047/2084] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:08:38 2022] PUT /identity/v3/projects/8c7b02690ff34b6fb25d25e755d06a97/users/734e54fa10cd4a12bc3df01de6fb604b/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:08:38.168339 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e96ccdbb-1ea3-4128-a159-42fae3864003 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e5dd9b04dcb4f2f86f1b8be00401ee3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:38.168897 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65f9d4-0d11-4521-b054-0558076a3450 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:38.169189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65f9d4-0d11-4521-b054-0558076a3450 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:38.169468 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65f9d4-0d11-4521-b054-0558076a3450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:38.178064 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e96ccdbb-1ea3-4128-a159-42fae3864003 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e5dd9b04dcb4f2f86f1b8be00401ee3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:38.197770 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ff65f9d4-0d11-4521-b054-0558076a3450 None None] MFA Rules not processed for user `734e54fa10cd4a12bc3df01de6fb604b`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:38.199036 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ff65f9d4-0d11-4521-b054-0558076a3450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:38.234189 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e96ccdbb-1ea3-4128-a159-42fae3864003 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e5dd9b04dcb4f2f86f1b8be00401ee3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:38.243907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1038/2085] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:08:37 2022] DELETE /identity/v3/users/3e5dd9b04dcb4f2f86f1b8be00401ee3 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:08:38.250037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b329f6-54d4-4c33-ad69-b75d2b741598 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:38.250345 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b329f6-54d4-4c33-ad69-b75d2b741598 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:38.250684 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b329f6-54d4-4c33-ad69-b75d2b741598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:38.284942 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-42b329f6-54d4-4c33-ad69-b75d2b741598 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:38.286451 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-42b329f6-54d4-4c33-ad69-b75d2b741598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:38.320264 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1048/2086] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:38.334910 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65f9d4-0d11-4521-b054-0558076a3450 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:38.337315 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ff65f9d4-0d11-4521-b054-0558076a3450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:38.382104 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1039/2087] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:08:38.392129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b329f6-54d4-4c33-ad69-b75d2b741598 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:38.393763 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-42b329f6-54d4-4c33-ad69-b75d2b741598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:38.430323 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65f9d4-0d11-4521-b054-0558076a3450 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:38.457009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b329f6-54d4-4c33-ad69-b75d2b741598 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:38.461096 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8c9d3-34ce-4bf4-8eac-8ee885b90579 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:38.461803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8c9d3-34ce-4bf4-8eac-8ee885b90579 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:38.462103 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8c9d3-34ce-4bf4-8eac-8ee885b90579 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:38.462471 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc8c9d3-34ce-4bf4-8eac-8ee885b90579 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:38.476378 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc8c9d3-34ce-4bf4-8eac-8ee885b90579 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:38.477893 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc8c9d3-34ce-4bf4-8eac-8ee885b90579 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:38.489621 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1040/2088] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 20:08:38.490154 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:38.508225 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f5b09-0744-4545-b813-b149d3da1619 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:38.508921 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f5b09-0744-4545-b813-b149d3da1619 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:38.509325 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f5b09-0744-4545-b813-b149d3da1619 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:38.509611 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f5b09-0744-4545-b813-b149d3da1619 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:38.567098 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f5b09-0744-4545-b813-b149d3da1619 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:38.568855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f5b09-0744-4545-b813-b149d3da1619 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:38.578471 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1049/2089] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:08:39.161458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8c9d3-34ce-4bf4-8eac-8ee885b90579 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:39.163016 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7bc8c9d3-34ce-4bf4-8eac-8ee885b90579 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:39.177635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc8c9d3-34ce-4bf4-8eac-8ee885b90579 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:39.182467 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc39d1e-783c-4b8d-bd79-d120795463e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:39.183326 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc39d1e-783c-4b8d-bd79-d120795463e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:39.183610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc39d1e-783c-4b8d-bd79-d120795463e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:39.183885 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc39d1e-783c-4b8d-bd79-d120795463e7 admin admin] PATH_INFO: `/v3/projects/506ce043e11e4b39b4115e82f56d184f` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:39.185113 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc39d1e-783c-4b8d-bd79-d120795463e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=506ce043e11e4b39b4115e82f56d184f)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:39.187283 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc39d1e-783c-4b8d-bd79-d120795463e7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:39.403487 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ecc39d1e-783c-4b8d-bd79-d120795463e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '506ce043e11e4b39b4115e82f56d184f', 'request_id': 'req-ecc39d1e-783c-4b8d-bd79-d120795463e7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:39.407930 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ecc39d1e-783c-4b8d-bd79-d120795463e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '506ce043e11e4b39b4115e82f56d184f', 'request_id': 'req-ecc39d1e-783c-4b8d-bd79-d120795463e7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:39.416053 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ecc39d1e-783c-4b8d-bd79-d120795463e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '506ce043e11e4b39b4115e82f56d184f', 'request_id': 'req-ecc39d1e-783c-4b8d-bd79-d120795463e7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:39.420398 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:39.529568 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ecc39d1e-783c-4b8d-bd79-d120795463e7 admin admin] The token cache is being invalidate because project 506ce043e11e4b39b4115e82f56d184f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:08:39.529855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ecc39d1e-783c-4b8d-bd79-d120795463e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:39.530532 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ecc39d1e-783c-4b8d-bd79-d120795463e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:39.535068 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1041/2090] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:08:39 2022] DELETE /identity/v3/projects/506ce043e11e4b39b4115e82f56d184f => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:39.540594 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f5b09-0744-4545-b813-b149d3da1619 service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:39.554661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f5b09-0744-4545-b813-b149d3da1619 service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:39.617053 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f2fa5-a6c1-4b66-9e3e-b055cf23c013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:39.617933 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f2fa5-a6c1-4b66-9e3e-b055cf23c013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:39.618213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f2fa5-a6c1-4b66-9e3e-b055cf23c013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:39.618487 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f2fa5-a6c1-4b66-9e3e-b055cf23c013 admin admin] PATH_INFO: `/v3/users/af7015bb2cb3499da7b73b95db457c89` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:39.619671 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f2fa5-a6c1-4b66-9e3e-b055cf23c013 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af7015bb2cb3499da7b73b95db457c89)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:39.621709 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f2fa5-a6c1-4b66-9e3e-b055cf23c013 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:39.812439 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c64f2fa5-a6c1-4b66-9e3e-b055cf23c013 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af7015bb2cb3499da7b73b95db457c89'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:39.813079 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c64f2fa5-a6c1-4b66-9e3e-b055cf23c013 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af7015bb2cb3499da7b73b95db457c89'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:39.884025 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c64f2fa5-a6c1-4b66-9e3e-b055cf23c013 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af7015bb2cb3499da7b73b95db457c89'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:39.884682 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c64f2fa5-a6c1-4b66-9e3e-b055cf23c013 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af7015bb2cb3499da7b73b95db457c89'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:39.897424 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c64f2fa5-a6c1-4b66-9e3e-b055cf23c013 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af7015bb2cb3499da7b73b95db457c89'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:39.910274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1050/2091] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:08:39 2022] DELETE /identity/v3/users/af7015bb2cb3499da7b73b95db457c89 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:40.350883 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc39d1e-783c-4b8d-bd79-d120795463e7 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:40.350883 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ecc39d1e-783c-4b8d-bd79-d120795463e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:40.366955 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc39d1e-783c-4b8d-bd79-d120795463e7 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:40.454704 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a350021-b2c6-4f89-ba49-176fcf454104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:40.457100 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a350021-b2c6-4f89-ba49-176fcf454104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:40.457100 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a350021-b2c6-4f89-ba49-176fcf454104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:40.457100 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a350021-b2c6-4f89-ba49-176fcf454104 admin admin] PATH_INFO: `/v3/projects/b44fa59ba673426f9990fea70f71c8e7` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:40.457951 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a350021-b2c6-4f89-ba49-176fcf454104 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b44fa59ba673426f9990fea70f71c8e7)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:40.459880 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a350021-b2c6-4f89-ba49-176fcf454104 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:40.651986 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2a350021-b2c6-4f89-ba49-176fcf454104 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b44fa59ba673426f9990fea70f71c8e7', 'request_id': 'req-2a350021-b2c6-4f89-ba49-176fcf454104'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:40.661888 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2a350021-b2c6-4f89-ba49-176fcf454104 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b44fa59ba673426f9990fea70f71c8e7', 'request_id': 'req-2a350021-b2c6-4f89-ba49-176fcf454104'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:40.668566 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2a350021-b2c6-4f89-ba49-176fcf454104 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b44fa59ba673426f9990fea70f71c8e7', 'request_id': 'req-2a350021-b2c6-4f89-ba49-176fcf454104'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:40.742931 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2a350021-b2c6-4f89-ba49-176fcf454104 admin admin] The token cache is being invalidate because project b44fa59ba673426f9990fea70f71c8e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:08:40.743227 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2a350021-b2c6-4f89-ba49-176fcf454104 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:40.743891 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2a350021-b2c6-4f89-ba49-176fcf454104 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:40.745535 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1042/2092] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:08:40 2022] DELETE /identity/v3/projects/b44fa59ba673426f9990fea70f71c8e7 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:40.749465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f2fa5-a6c1-4b66-9e3e-b055cf23c013 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:40.750897 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c64f2fa5-a6c1-4b66-9e3e-b055cf23c013 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:40.764061 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f2fa5-a6c1-4b66-9e3e-b055cf23c013 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:40.769208 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:40.822765 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf9d58-4f51-4b92-a7e0-5a0066a4e34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:40.823631 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf9d58-4f51-4b92-a7e0-5a0066a4e34a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:40.823909 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf9d58-4f51-4b92-a7e0-5a0066a4e34a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:40.824181 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf9d58-4f51-4b92-a7e0-5a0066a4e34a admin admin] PATH_INFO: `/v3/users/55007a692d954c0e991a3770a7f6c0a1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:40.825396 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf9d58-4f51-4b92-a7e0-5a0066a4e34a admin admin] RBAC: Authorizing `identity:delete_user(user_id=55007a692d954c0e991a3770a7f6c0a1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:40.827471 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf9d58-4f51-4b92-a7e0-5a0066a4e34a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:41.068997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dfaf9d58-4f51-4b92-a7e0-5a0066a4e34a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55007a692d954c0e991a3770a7f6c0a1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:41.069649 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dfaf9d58-4f51-4b92-a7e0-5a0066a4e34a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55007a692d954c0e991a3770a7f6c0a1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:41.228569 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dfaf9d58-4f51-4b92-a7e0-5a0066a4e34a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55007a692d954c0e991a3770a7f6c0a1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:41.229057 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dfaf9d58-4f51-4b92-a7e0-5a0066a4e34a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55007a692d954c0e991a3770a7f6c0a1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:41.242321 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dfaf9d58-4f51-4b92-a7e0-5a0066a4e34a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55007a692d954c0e991a3770a7f6c0a1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:41.255010 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1051/2093] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:08:40 2022] DELETE /identity/v3/users/55007a692d954c0e991a3770a7f6c0a1 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:41.524251 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73929a3a-36fd-450c-9bd8-eab0678ce76c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:41.524802 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73929a3a-36fd-450c-9bd8-eab0678ce76c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:41.524913 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73929a3a-36fd-450c-9bd8-eab0678ce76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:41.548641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-73929a3a-36fd-450c-9bd8-eab0678ce76c None None] MFA Rules not processed for user `eafea4329d6243459dd3a6add1e91a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:41.549861 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-73929a3a-36fd-450c-9bd8-eab0678ce76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:41.622674 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1043/2094] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:41.704185 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf9d58-4f51-4b92-a7e0-5a0066a4e34a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:41.705785 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-dfaf9d58-4f51-4b92-a7e0-5a0066a4e34a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:41.793274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf9d58-4f51-4b92-a7e0-5a0066a4e34a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:41.798978 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae63ddf-351a-41ff-9407-70efe3f7babe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:41.799947 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae63ddf-351a-41ff-9407-70efe3f7babe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:41.800334 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae63ddf-351a-41ff-9407-70efe3f7babe admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:41.800682 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae63ddf-351a-41ff-9407-70efe3f7babe admin admin] PATH_INFO: `/v3/projects/af1527b3e173427e893fff95b811fc30` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:41.802543 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae63ddf-351a-41ff-9407-70efe3f7babe admin admin] RBAC: Authorizing `identity:delete_project(project_id=af1527b3e173427e893fff95b811fc30)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:41.805559 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae63ddf-351a-41ff-9407-70efe3f7babe admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:41.918643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:41.932081 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3ae63ddf-351a-41ff-9407-70efe3f7babe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af1527b3e173427e893fff95b811fc30', 'request_id': 'req-3ae63ddf-351a-41ff-9407-70efe3f7babe'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:42.029452 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3ae63ddf-351a-41ff-9407-70efe3f7babe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af1527b3e173427e893fff95b811fc30', 'request_id': 'req-3ae63ddf-351a-41ff-9407-70efe3f7babe'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:42.040196 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3ae63ddf-351a-41ff-9407-70efe3f7babe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af1527b3e173427e893fff95b811fc30', 'request_id': 'req-3ae63ddf-351a-41ff-9407-70efe3f7babe'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:42.049826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73929a3a-36fd-450c-9bd8-eab0678ce76c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:42.052038 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-73929a3a-36fd-450c-9bd8-eab0678ce76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:42.053516 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3ae63ddf-351a-41ff-9407-70efe3f7babe admin admin] The token cache is being invalidate because project af1527b3e173427e893fff95b811fc30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:08:42.054057 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3ae63ddf-351a-41ff-9407-70efe3f7babe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:42.057847 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3ae63ddf-351a-41ff-9407-70efe3f7babe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:42.061108 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1052/2095] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:08:41 2022] DELETE /identity/v3/projects/af1527b3e173427e893fff95b811fc30 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:42.066313 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ed155-4680-4c64-84a6-acd432037021 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:42.066594 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ed155-4680-4c64-84a6-acd432037021 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:42.066862 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ed155-4680-4c64-84a6-acd432037021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:42.072337 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73929a3a-36fd-450c-9bd8-eab0678ce76c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:42.093735 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:42.104597 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ba2ed155-4680-4c64-84a6-acd432037021 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:42.105838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ba2ed155-4680-4c64-84a6-acd432037021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:42.143899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9862a9-6b1c-4868-b5ac-d7abbb58e1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:42.145027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9862a9-6b1c-4868-b5ac-d7abbb58e1c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:42.145408 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9862a9-6b1c-4868-b5ac-d7abbb58e1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:42.145878 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9862a9-6b1c-4868-b5ac-d7abbb58e1c6 admin admin] PATH_INFO: `/v3/users/734e54fa10cd4a12bc3df01de6fb604b` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:42.152659 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9862a9-6b1c-4868-b5ac-d7abbb58e1c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=734e54fa10cd4a12bc3df01de6fb604b)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:42.155529 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9862a9-6b1c-4868-b5ac-d7abbb58e1c6 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:42.181457 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1053/2096] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:08:42.189492 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75536889-5bf5-46fc-aa20-24bf02f8b0ac None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:42.189778 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75536889-5bf5-46fc-aa20-24bf02f8b0ac None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:42.190053 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75536889-5bf5-46fc-aa20-24bf02f8b0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:42.210359 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-75536889-5bf5-46fc-aa20-24bf02f8b0ac None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:42.211587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-75536889-5bf5-46fc-aa20-24bf02f8b0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:42.220421 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1054/2097] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:08:42.224446 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75536889-5bf5-46fc-aa20-24bf02f8b0ac None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:42.233830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75536889-5bf5-46fc-aa20-24bf02f8b0ac None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:42.241051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea150a-55ac-4d2c-9cfd-b75a3e81444a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:42.241714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea150a-55ac-4d2c-9cfd-b75a3e81444a admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:42.241997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea150a-55ac-4d2c-9cfd-b75a3e81444a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:42.243262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aea150a-55ac-4d2c-9cfd-b75a3e81444a admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:42.243262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea150a-55ac-4d2c-9cfd-b75a3e81444a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:08:42.243262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea150a-55ac-4d2c-9cfd-b75a3e81444a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:42.245017 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aea150a-55ac-4d2c-9cfd-b75a3e81444a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:42.257944 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1055/2098] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 20:08:42.262253 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea150a-55ac-4d2c-9cfd-b75a3e81444a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:42.272721 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aea150a-55ac-4d2c-9cfd-b75a3e81444a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:42.278038 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a00271-8a29-4c19-aa79-3c33d2d6353e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:42.278331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a00271-8a29-4c19-aa79-3c33d2d6353e admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:42.278650 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a00271-8a29-4c19-aa79-3c33d2d6353e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:42.278906 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a00271-8a29-4c19-aa79-3c33d2d6353e admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:42.279625 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a00271-8a29-4c19-aa79-3c33d2d6353e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:42.281605 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a00271-8a29-4c19-aa79-3c33d2d6353e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:42.380953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ff9862a9-6b1c-4868-b5ac-d7abbb58e1c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '734e54fa10cd4a12bc3df01de6fb604b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:42.381784 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ff9862a9-6b1c-4868-b5ac-d7abbb58e1c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '734e54fa10cd4a12bc3df01de6fb604b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:42.397607 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ff9862a9-6b1c-4868-b5ac-d7abbb58e1c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '734e54fa10cd4a12bc3df01de6fb604b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:42.400639 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1056/2099] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:08:42 2022] POST /identity/v3/projects => generated 373 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:08:42.404836 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a00271-8a29-4c19-aa79-3c33d2d6353e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:42.410099 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ff9862a9-6b1c-4868-b5ac-d7abbb58e1c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '734e54fa10cd4a12bc3df01de6fb604b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:42.471289 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a00271-8a29-4c19-aa79-3c33d2d6353e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:42.475895 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ec6dd-0e2b-4dd2-b2bf-43e47408d059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:42.476581 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ec6dd-0e2b-4dd2-b2bf-43e47408d059 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:42.476862 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ec6dd-0e2b-4dd2-b2bf-43e47408d059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:42.477223 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ec6dd-0e2b-4dd2-b2bf-43e47408d059 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:42.477914 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ec6dd-0e2b-4dd2-b2bf-43e47408d059 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:42.479858 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ec6dd-0e2b-4dd2-b2bf-43e47408d059 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:42.491475 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ff9862a9-6b1c-4868-b5ac-d7abbb58e1c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '734e54fa10cd4a12bc3df01de6fb604b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:42.499687 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1044/2100] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:08:42 2022] DELETE /identity/v3/users/734e54fa10cd4a12bc3df01de6fb604b => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:42.504732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a0cbf-392c-4a15-95f7-07b023b35d67 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:42.505197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a0cbf-392c-4a15-95f7-07b023b35d67 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:42.505488 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a0cbf-392c-4a15-95f7-07b023b35d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:42.524347 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e92a0cbf-392c-4a15-95f7-07b023b35d67 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:42.525568 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e92a0cbf-392c-4a15-95f7-07b023b35d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:42.586410 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1045/2101] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:42.598403 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a0cbf-392c-4a15-95f7-07b023b35d67 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:42.604863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1057/2102] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:08:42 2022] POST /identity/v3/users => generated 287 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:08:42.606196 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e92a0cbf-392c-4a15-95f7-07b023b35d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:42.609731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ec6dd-0e2b-4dd2-b2bf-43e47408d059 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:42.622080 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8e3ec6dd-0e2b-4dd2-b2bf-43e47408d059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:42.635420 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ec6dd-0e2b-4dd2-b2bf-43e47408d059 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:42.640639 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4ad74-f0e5-4834-994b-c4de3fa55e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:42.641397 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4ad74-f0e5-4834-994b-c4de3fa55e4c admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:42.641745 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4ad74-f0e5-4834-994b-c4de3fa55e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:42.642087 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4ad74-f0e5-4834-994b-c4de3fa55e4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:42.642890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc4ad74-f0e5-4834-994b-c4de3fa55e4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:42.644702 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc4ad74-f0e5-4834-994b-c4de3fa55e4c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:42.646152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:42.655226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1058/2103] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:08:42.659740 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4ad74-f0e5-4834-994b-c4de3fa55e4c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:42.669452 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4ad74-f0e5-4834-994b-c4de3fa55e4c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:42.674680 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ca715-b3cf-4541-81e5-ddc3e8022c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:42.675317 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ca715-b3cf-4541-81e5-ddc3e8022c09 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:42.675598 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ca715-b3cf-4541-81e5-ddc3e8022c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:42.675868 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ca715-b3cf-4541-81e5-ddc3e8022c09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:42.676614 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ca715-b3cf-4541-81e5-ddc3e8022c09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:42.677121 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a0cbf-392c-4a15-95f7-07b023b35d67 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:42.678210 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ca715-b3cf-4541-81e5-ddc3e8022c09 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:42.680801 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9918d8fe-52e6-49c1-87a4-d1aabd2a5475 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:42.681461 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918d8fe-52e6-49c1-87a4-d1aabd2a5475 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:42.681835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918d8fe-52e6-49c1-87a4-d1aabd2a5475 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:42.682177 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918d8fe-52e6-49c1-87a4-d1aabd2a5475 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:42.689025 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1059/2104] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:08:42.693144 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ca715-b3cf-4541-81e5-ddc3e8022c09 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:42.693884 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9918d8fe-52e6-49c1-87a4-d1aabd2a5475 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:42.695348 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9918d8fe-52e6-49c1-87a4-d1aabd2a5475 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:42.703038 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ca715-b3cf-4541-81e5-ddc3e8022c09 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:42.705637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1046/2105] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:08:42.708929 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cbcf51-f47f-455c-9f01-ac6485f57558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:42.709830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbcf51-f47f-455c-9f01-ac6485f57558 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:42.710107 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbcf51-f47f-455c-9f01-ac6485f57558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:42.710379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbcf51-f47f-455c-9f01-ac6485f57558 admin admin] PATH_INFO: `/v3/projects/df7d693ea21441559f4db447f5414f69/users/a6a6e5f6e5f24ba89ca4895aebe158f8/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:42.728772 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cbcf51-f47f-455c-9f01-ac6485f57558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df7d693ea21441559f4db447f5414f69, user_id=a6a6e5f6e5f24ba89ca4895aebe158f8, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:42.732181 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cbcf51-f47f-455c-9f01-ac6485f57558 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:42.824654 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1060/2106] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:08:42 2022] PUT /identity/v3/projects/df7d693ea21441559f4db447f5414f69/users/a6a6e5f6e5f24ba89ca4895aebe158f8/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:08:42.830349 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782e2b2-7ce8-44fb-a046-78f2def60354 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:42.830706 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782e2b2-7ce8-44fb-a046-78f2def60354 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:42.831053 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782e2b2-7ce8-44fb-a046-78f2def60354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:42.857593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e782e2b2-7ce8-44fb-a046-78f2def60354 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:42.858794 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e782e2b2-7ce8-44fb-a046-78f2def60354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:42.942782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1047/2107] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:08:42.955496 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cbcf51-f47f-455c-9f01-ac6485f57558 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:42.973756 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c1cbcf51-f47f-455c-9f01-ac6485f57558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:43.022713 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:43.103516 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cbcf51-f47f-455c-9f01-ac6485f57558 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:43.112117 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cabe6c7-6ed6-4935-a231-81ef16e4c9a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:43.112942 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabe6c7-6ed6-4935-a231-81ef16e4c9a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:43.113226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabe6c7-6ed6-4935-a231-81ef16e4c9a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:43.113503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabe6c7-6ed6-4935-a231-81ef16e4c9a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:43.124212 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cabe6c7-6ed6-4935-a231-81ef16e4c9a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:43.126459 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cabe6c7-6ed6-4935-a231-81ef16e4c9a4 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:43.132730 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782e2b2-7ce8-44fb-a046-78f2def60354 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:43.134917 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e782e2b2-7ce8-44fb-a046-78f2def60354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:43.143377 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1061/2108] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:08:43.158030 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782e2b2-7ce8-44fb-a046-78f2def60354 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:43.163323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe03fc-e91a-46ee-ac46-034f9693b229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:43.164268 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe03fc-e91a-46ee-ac46-034f9693b229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:43.164863 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe03fc-e91a-46ee-ac46-034f9693b229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:43.164863 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe03fc-e91a-46ee-ac46-034f9693b229 admin admin] PATH_INFO: `/v3/users/6f53906c1b1b4de4b2c39a4c1551f68d` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:43.166107 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe03fc-e91a-46ee-ac46-034f9693b229 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f53906c1b1b4de4b2c39a4c1551f68d)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:43.168334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afe03fc-e91a-46ee-ac46-034f9693b229 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:43.269964 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cabe6c7-6ed6-4935-a231-81ef16e4c9a4 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:43.272105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2cabe6c7-6ed6-4935-a231-81ef16e4c9a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:43.311231 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cabe6c7-6ed6-4935-a231-81ef16e4c9a4 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:43.320356 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f44f2-b105-4610-b3b4-5cc787d15bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:43.321907 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f44f2-b105-4610-b3b4-5cc787d15bdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:43.321907 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f44f2-b105-4610-b3b4-5cc787d15bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:43.322260 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f44f2-b105-4610-b3b4-5cc787d15bdc admin admin] PATH_INFO: `/v3/projects/8c7b02690ff34b6fb25d25e755d06a97` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:43.323594 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f44f2-b105-4610-b3b4-5cc787d15bdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c7b02690ff34b6fb25d25e755d06a97)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:43.326187 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f44f2-b105-4610-b3b4-5cc787d15bdc admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:43.379386 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9afe03fc-e91a-46ee-ac46-034f9693b229 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f53906c1b1b4de4b2c39a4c1551f68d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:43.380119 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9afe03fc-e91a-46ee-ac46-034f9693b229 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f53906c1b1b4de4b2c39a4c1551f68d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:43.401720 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9afe03fc-e91a-46ee-ac46-034f9693b229 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f53906c1b1b4de4b2c39a4c1551f68d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:43.417361 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9afe03fc-e91a-46ee-ac46-034f9693b229 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f53906c1b1b4de4b2c39a4c1551f68d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:43.470713 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e94f44f2-b105-4610-b3b4-5cc787d15bdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c7b02690ff34b6fb25d25e755d06a97', 'request_id': 'req-e94f44f2-b105-4610-b3b4-5cc787d15bdc'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:43.536735 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9afe03fc-e91a-46ee-ac46-034f9693b229 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f53906c1b1b4de4b2c39a4c1551f68d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:43.544549 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1048/2109] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:08:43 2022] DELETE /identity/v3/users/6f53906c1b1b4de4b2c39a4c1551f68d => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:08:43.551167 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b09d5-c439-429e-bf86-a6ec55159e62 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:43.551167 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b09d5-c439-429e-bf86-a6ec55159e62 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:43.551167 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b09d5-c439-429e-bf86-a6ec55159e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:43.576291 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b50b09d5-c439-429e-bf86-a6ec55159e62 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:43.577489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b50b09d5-c439-429e-bf86-a6ec55159e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:43.633083 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e94f44f2-b105-4610-b3b4-5cc787d15bdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c7b02690ff34b6fb25d25e755d06a97', 'request_id': 'req-e94f44f2-b105-4610-b3b4-5cc787d15bdc'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:43.637074 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e94f44f2-b105-4610-b3b4-5cc787d15bdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c7b02690ff34b6fb25d25e755d06a97', 'request_id': 'req-e94f44f2-b105-4610-b3b4-5cc787d15bdc'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:43.657745 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:43.659294 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e94f44f2-b105-4610-b3b4-5cc787d15bdc admin admin] The token cache is being invalidate because project 8c7b02690ff34b6fb25d25e755d06a97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:08:43.659593 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e94f44f2-b105-4610-b3b4-5cc787d15bdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:43.660523 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e94f44f2-b105-4610-b3b4-5cc787d15bdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:43.663076 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1062/2110] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:08:43 2022] DELETE /identity/v3/projects/8c7b02690ff34b6fb25d25e755d06a97 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:43.675451 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00750d-5f11-4dde-a5b9-c8e40e46d687 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:43.675743 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00750d-5f11-4dde-a5b9-c8e40e46d687 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:43.677345 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00750d-5f11-4dde-a5b9-c8e40e46d687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:43.686095 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1049/2111] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:43.697105 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b09d5-c439-429e-bf86-a6ec55159e62 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:43.698618 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b50b09d5-c439-429e-bf86-a6ec55159e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:43.713889 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-4f00750d-5f11-4dde-a5b9-c8e40e46d687 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:43.715632 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4f00750d-5f11-4dde-a5b9-c8e40e46d687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:43.747956 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1063/2112] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:43.757237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298739a2-b021-4da0-a55d-5be26888af56 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:43.757656 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298739a2-b021-4da0-a55d-5be26888af56 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:43.757812 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298739a2-b021-4da0-a55d-5be26888af56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:43.788567 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-298739a2-b021-4da0-a55d-5be26888af56 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:43.790067 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-298739a2-b021-4da0-a55d-5be26888af56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:43.803796 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1064/2113] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:43.803992 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b09d5-c439-429e-bf86-a6ec55159e62 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:43.804982 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298739a2-b021-4da0-a55d-5be26888af56 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:43.809994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524e134-c866-4dd0-aeda-53a8ed587034 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:43.811152 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524e134-c866-4dd0-aeda-53a8ed587034 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:43.811813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524e134-c866-4dd0-aeda-53a8ed587034 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:43.812188 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524e134-c866-4dd0-aeda-53a8ed587034 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:43.817931 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298739a2-b021-4da0-a55d-5be26888af56 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:43.830220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7524e134-c866-4dd0-aeda-53a8ed587034 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:43.832194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7524e134-c866-4dd0-aeda-53a8ed587034 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:43.841591 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5532ea8-9987-479d-ac42-54f3214d9c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:43.844425 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5532ea8-9987-479d-ac42-54f3214d9c0a admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:43.844725 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5532ea8-9987-479d-ac42-54f3214d9c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:43.845023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5532ea8-9987-479d-ac42-54f3214d9c0a admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:43.845763 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5532ea8-9987-479d-ac42-54f3214d9c0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:08:43.846427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5532ea8-9987-479d-ac42-54f3214d9c0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:43.849625 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5532ea8-9987-479d-ac42-54f3214d9c0a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:43.862401 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1050/2114] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:08:43.886061 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1065/2115] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:08:43.891045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524e134-c866-4dd0-aeda-53a8ed587034 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:43.905533 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524e134-c866-4dd0-aeda-53a8ed587034 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:43.923552 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b4f43-2fcc-4ee8-afc6-3f3c4ab2bffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:43.926961 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b4f43-2fcc-4ee8-afc6-3f3c4ab2bffc admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:43.926961 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b4f43-2fcc-4ee8-afc6-3f3c4ab2bffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:43.926961 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75b4f43-2fcc-4ee8-afc6-3f3c4ab2bffc admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:43.926961 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b4f43-2fcc-4ee8-afc6-3f3c4ab2bffc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:43.929277 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75b4f43-2fcc-4ee8-afc6-3f3c4ab2bffc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:44.037389 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1051/2116] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:08:43 2022] POST /identity/v3/projects => generated 365 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:44.041258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5532ea8-9987-479d-ac42-54f3214d9c0a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:44.053710 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5532ea8-9987-479d-ac42-54f3214d9c0a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:44.077865 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:44.129144 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac8206-c6a0-4257-9287-00f54ebab454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:44.129879 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac8206-c6a0-4257-9287-00f54ebab454 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:44.130293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac8206-c6a0-4257-9287-00f54ebab454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:44.130576 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac8206-c6a0-4257-9287-00f54ebab454 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:44.131334 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac8206-c6a0-4257-9287-00f54ebab454 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:44.133558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac8206-c6a0-4257-9287-00f54ebab454 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:44.268859 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1066/2117] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:08:44 2022] POST /identity/v3/users => generated 283 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:44.272967 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b4f43-2fcc-4ee8-afc6-3f3c4ab2bffc admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:44.287593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75b4f43-2fcc-4ee8-afc6-3f3c4ab2bffc admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:44.294298 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5dac0-9556-4c46-82f3-b5960175ffa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:44.294999 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5dac0-9556-4c46-82f3-b5960175ffa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:44.295686 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5dac0-9556-4c46-82f3-b5960175ffa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:44.295686 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5dac0-9556-4c46-82f3-b5960175ffa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:44.296325 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c5dac0-9556-4c46-82f3-b5960175ffa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:44.298615 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c5dac0-9556-4c46-82f3-b5960175ffa9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:44.311348 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1052/2118] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:44.315156 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac8206-c6a0-4257-9287-00f54ebab454 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:44.330369 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac8206-c6a0-4257-9287-00f54ebab454 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:44.335275 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59030af-e72c-49d0-984f-7182e0c2afd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:44.335891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59030af-e72c-49d0-984f-7182e0c2afd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:44.336179 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59030af-e72c-49d0-984f-7182e0c2afd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:44.336496 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59030af-e72c-49d0-984f-7182e0c2afd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:44.337245 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59030af-e72c-49d0-984f-7182e0c2afd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:44.338898 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59030af-e72c-49d0-984f-7182e0c2afd8 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:44.353836 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1067/2119] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:44.358228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5dac0-9556-4c46-82f3-b5960175ffa9 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:44.378863 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5dac0-9556-4c46-82f3-b5960175ffa9 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:44.384483 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca09fd-7946-419e-84cb-19dd7c8a7502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:44.385578 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca09fd-7946-419e-84cb-19dd7c8a7502 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:44.386132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca09fd-7946-419e-84cb-19dd7c8a7502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:44.388389 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca09fd-7946-419e-84cb-19dd7c8a7502 admin admin] PATH_INFO: `/v3/projects/06fdc22c735647dd802b7d19b590fe30/users/d26da25b32cd434795aa19ec60e031bc/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:44.410658 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca09fd-7946-419e-84cb-19dd7c8a7502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06fdc22c735647dd802b7d19b590fe30, user_id=d26da25b32cd434795aa19ec60e031bc, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:44.413430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca09fd-7946-419e-84cb-19dd7c8a7502 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:44.520614 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481a057-3585-4aa3-803a-c568bfa30adc None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:44.520887 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481a057-3585-4aa3-803a-c568bfa30adc None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:44.521097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481a057-3585-4aa3-803a-c568bfa30adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:44.523271 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1053/2120] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:08:44 2022] PUT /identity/v3/projects/06fdc22c735647dd802b7d19b590fe30/users/d26da25b32cd434795aa19ec60e031bc/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:44.553719 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-6481a057-3585-4aa3-803a-c568bfa30adc None None] MFA Rules not processed for user `d26da25b32cd434795aa19ec60e031bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:44.555143 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6481a057-3585-4aa3-803a-c568bfa30adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:44.623817 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1068/2121] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:44 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:44.636229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca09fd-7946-419e-84cb-19dd7c8a7502 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:44.637700 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-46ca09fd-7946-419e-84cb-19dd7c8a7502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:44.671166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481a057-3585-4aa3-803a-c568bfa30adc None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:44.671166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6481a057-3585-4aa3-803a-c568bfa30adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:44.710005 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca09fd-7946-419e-84cb-19dd7c8a7502 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:44.714567 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c34f3-dd99-4537-a1f5-1d5c54d6c8f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:44.716139 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c34f3-dd99-4537-a1f5-1d5c54d6c8f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:44.716438 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c34f3-dd99-4537-a1f5-1d5c54d6c8f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:44.716796 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c34f3-dd99-4537-a1f5-1d5c54d6c8f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:44.728028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c34f3-dd99-4537-a1f5-1d5c54d6c8f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:44.729644 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c34f3-dd99-4537-a1f5-1d5c54d6c8f9 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:44.742766 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1054/2122] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:44 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:08:44.763867 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481a057-3585-4aa3-803a-c568bfa30adc None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:44.769844 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e52a8f-1ba7-40b7-a568-f948ba9f077d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:44.771509 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e52a8f-1ba7-40b7-a568-f948ba9f077d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:44.771793 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e52a8f-1ba7-40b7-a568-f948ba9f077d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:44.772067 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e52a8f-1ba7-40b7-a568-f948ba9f077d admin admin] PATH_INFO: `/v3/projects/cbb4cea2394847a4994be78222d4aa83` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:44.773531 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e52a8f-1ba7-40b7-a568-f948ba9f077d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbb4cea2394847a4994be78222d4aa83)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:44.776186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e52a8f-1ba7-40b7-a568-f948ba9f077d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:44.921328 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0d7d1-3834-477c-b788-efe920c3b9bb None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:44.921631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0d7d1-3834-477c-b788-efe920c3b9bb None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:44.921977 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0d7d1-3834-477c-b788-efe920c3b9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:44.938596 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b8e52a8f-1ba7-40b7-a568-f948ba9f077d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbb4cea2394847a4994be78222d4aa83', 'request_id': 'req-b8e52a8f-1ba7-40b7-a568-f948ba9f077d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:44.962132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f6b0d7d1-3834-477c-b788-efe920c3b9bb None None] MFA Rules not processed for user `a6a6e5f6e5f24ba89ca4895aebe158f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:44.963312 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f6b0d7d1-3834-477c-b788-efe920c3b9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:45.014028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:45.027731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b8e52a8f-1ba7-40b7-a568-f948ba9f077d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbb4cea2394847a4994be78222d4aa83', 'request_id': 'req-b8e52a8f-1ba7-40b7-a568-f948ba9f077d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:45.034585 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b8e52a8f-1ba7-40b7-a568-f948ba9f077d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbb4cea2394847a4994be78222d4aa83', 'request_id': 'req-b8e52a8f-1ba7-40b7-a568-f948ba9f077d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:45.046270 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b8e52a8f-1ba7-40b7-a568-f948ba9f077d admin admin] The token cache is being invalidate because project cbb4cea2394847a4994be78222d4aa83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:08:45.046564 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b8e52a8f-1ba7-40b7-a568-f948ba9f077d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:45.047206 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b8e52a8f-1ba7-40b7-a568-f948ba9f077d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:45.049082 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1069/2123] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:08:44 2022] DELETE /identity/v3/projects/cbb4cea2394847a4994be78222d4aa83 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:08:45.053167 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e52a8f-1ba7-40b7-a568-f948ba9f077d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:45.054925 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b8e52a8f-1ba7-40b7-a568-f948ba9f077d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:45.057017 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1055/2124] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:45.068371 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b0d7d1-3834-477c-b788-efe920c3b9bb None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:45.069885 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f6b0d7d1-3834-477c-b788-efe920c3b9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:45.136382 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b0d7d1-3834-477c-b788-efe920c3b9bb None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:45.140370 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e52a8f-1ba7-40b7-a568-f948ba9f077d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:45.140950 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5fa00b-0009-4248-a5a1-3dede2d291d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:45.142216 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fa00b-0009-4248-a5a1-3dede2d291d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:45.142505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fa00b-0009-4248-a5a1-3dede2d291d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:45.142782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fa00b-0009-4248-a5a1-3dede2d291d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:45.144686 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8a5fa00b-0009-4248-a5a1-3dede2d291d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:45.147583 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccdfce0-31d0-40bc-adcd-652cb672d04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:45.148661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdfce0-31d0-40bc-adcd-652cb672d04d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:45.149107 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdfce0-31d0-40bc-adcd-652cb672d04d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:45.149385 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccdfce0-31d0-40bc-adcd-652cb672d04d admin admin] PATH_INFO: `/v3/users/243793d3d042416ab46109cf4a2d5088` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:45.150577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccdfce0-31d0-40bc-adcd-652cb672d04d admin admin] RBAC: Authorizing `identity:delete_user(user_id=243793d3d042416ab46109cf4a2d5088)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:45.152589 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccdfce0-31d0-40bc-adcd-652cb672d04d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:45.156031 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5fa00b-0009-4248-a5a1-3dede2d291d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:45.157545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5fa00b-0009-4248-a5a1-3dede2d291d4 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:45.166469 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1056/2125] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 20:08:45.314086 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:45.356121 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dccdfce0-31d0-40bc-adcd-652cb672d04d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '243793d3d042416ab46109cf4a2d5088'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:45.356979 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dccdfce0-31d0-40bc-adcd-652cb672d04d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '243793d3d042416ab46109cf4a2d5088'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:45.433370 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dccdfce0-31d0-40bc-adcd-652cb672d04d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '243793d3d042416ab46109cf4a2d5088'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:45.434151 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dccdfce0-31d0-40bc-adcd-652cb672d04d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '243793d3d042416ab46109cf4a2d5088'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:45.452039 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dccdfce0-31d0-40bc-adcd-652cb672d04d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '243793d3d042416ab46109cf4a2d5088'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:45.470116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1070/2126] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:08:45 2022] DELETE /identity/v3/users/243793d3d042416ab46109cf4a2d5088 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:08:45.885739 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5fa00b-0009-4248-a5a1-3dede2d291d4 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:45.897027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5fa00b-0009-4248-a5a1-3dede2d291d4 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:45.956614 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb6750-3195-46e6-a740-249cde681276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:45.957548 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb6750-3195-46e6-a740-249cde681276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:45.957831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb6750-3195-46e6-a740-249cde681276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:45.958254 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb6750-3195-46e6-a740-249cde681276 admin admin] PATH_INFO: `/v3/projects/161fdbb8c7ae4ab89f4df22482076b9e` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:45.959475 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb6750-3195-46e6-a740-249cde681276 admin admin] RBAC: Authorizing `identity:delete_project(project_id=161fdbb8c7ae4ab89f4df22482076b9e)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:45.961484 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb6750-3195-46e6-a740-249cde681276 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:46.134466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-43bb6750-3195-46e6-a740-249cde681276 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '161fdbb8c7ae4ab89f4df22482076b9e', 'request_id': 'req-43bb6750-3195-46e6-a740-249cde681276'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:46.141892 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-43bb6750-3195-46e6-a740-249cde681276 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '161fdbb8c7ae4ab89f4df22482076b9e', 'request_id': 'req-43bb6750-3195-46e6-a740-249cde681276'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:46.152544 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-43bb6750-3195-46e6-a740-249cde681276 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '161fdbb8c7ae4ab89f4df22482076b9e', 'request_id': 'req-43bb6750-3195-46e6-a740-249cde681276'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:46.274098 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-43bb6750-3195-46e6-a740-249cde681276 admin admin] The token cache is being invalidate because project 161fdbb8c7ae4ab89f4df22482076b9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:08:46.274491 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-43bb6750-3195-46e6-a740-249cde681276 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:46.275896 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-43bb6750-3195-46e6-a740-249cde681276 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:46.278385 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1057/2127] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:08:45 2022] DELETE /identity/v3/projects/161fdbb8c7ae4ab89f4df22482076b9e => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:46.284741 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c268366-43f9-4a0e-89f4-7cac6ce99e98 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:46.285226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c268366-43f9-4a0e-89f4-7cac6ce99e98 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:46.285414 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c268366-43f9-4a0e-89f4-7cac6ce99e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:46.303842 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-4c268366-43f9-4a0e-89f4-7cac6ce99e98 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:46.305167 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4c268366-43f9-4a0e-89f4-7cac6ce99e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:46.365926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1071/2128] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:46.375255 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a919e2-1050-4103-9e2e-aaa35f8f8f55 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:46.376068 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a919e2-1050-4103-9e2e-aaa35f8f8f55 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:46.378083 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a919e2-1050-4103-9e2e-aaa35f8f8f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:46.391004 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:46.397011 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f6a919e2-1050-4103-9e2e-aaa35f8f8f55 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:46.398269 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f6a919e2-1050-4103-9e2e-aaa35f8f8f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:46.406237 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1058/2129] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:08:46.410414 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c268366-43f9-4a0e-89f4-7cac6ce99e98 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:46.420571 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c268366-43f9-4a0e-89f4-7cac6ce99e98 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:46.425170 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897660aa-0ac8-4738-9a9a-4d4bd75686a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:46.425876 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897660aa-0ac8-4738-9a9a-4d4bd75686a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:46.426157 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897660aa-0ac8-4738-9a9a-4d4bd75686a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:46.426430 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897660aa-0ac8-4738-9a9a-4d4bd75686a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:46.427157 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897660aa-0ac8-4738-9a9a-4d4bd75686a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:08:46.427465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897660aa-0ac8-4738-9a9a-4d4bd75686a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:46.429232 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897660aa-0ac8-4738-9a9a-4d4bd75686a5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:46.442881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1072/2130] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:08:46.447414 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a919e2-1050-4103-9e2e-aaa35f8f8f55 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:46.459935 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a919e2-1050-4103-9e2e-aaa35f8f8f55 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:46.465015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71e6a9-42b5-48cd-bb08-5c175c0ab7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:46.465619 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71e6a9-42b5-48cd-bb08-5c175c0ab7dc admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:46.465891 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71e6a9-42b5-48cd-bb08-5c175c0ab7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:46.466163 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71e6a9-42b5-48cd-bb08-5c175c0ab7dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:46.466854 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e71e6a9-42b5-48cd-bb08-5c175c0ab7dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:46.468828 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e71e6a9-42b5-48cd-bb08-5c175c0ab7dc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:46.611602 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1059/2131] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:08:46 2022] POST /identity/v3/projects => generated 383 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:46.616891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897660aa-0ac8-4738-9a9a-4d4bd75686a5 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:46.628413 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897660aa-0ac8-4738-9a9a-4d4bd75686a5 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:46.690413 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c19eb1-5733-48ae-98b9-1cea28d93047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:46.691141 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c19eb1-5733-48ae-98b9-1cea28d93047 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:46.691424 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c19eb1-5733-48ae-98b9-1cea28d93047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:46.691700 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c19eb1-5733-48ae-98b9-1cea28d93047 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:46.692453 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c19eb1-5733-48ae-98b9-1cea28d93047 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:46.694470 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c19eb1-5733-48ae-98b9-1cea28d93047 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:46.791902 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1073/2132] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:08:46 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:46.796072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71e6a9-42b5-48cd-bb08-5c175c0ab7dc admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:46.807382 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71e6a9-42b5-48cd-bb08-5c175c0ab7dc admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:46.812220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071439b6-aecd-4b10-8b31-d43d04b9b6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:46.812998 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071439b6-aecd-4b10-8b31-d43d04b9b6a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:46.813856 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071439b6-aecd-4b10-8b31-d43d04b9b6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:46.814134 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071439b6-aecd-4b10-8b31-d43d04b9b6a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:46.814892 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071439b6-aecd-4b10-8b31-d43d04b9b6a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:46.816529 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071439b6-aecd-4b10-8b31-d43d04b9b6a6 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:46.828464 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1060/2133] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:46.833444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c19eb1-5733-48ae-98b9-1cea28d93047 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:46.844225 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c19eb1-5733-48ae-98b9-1cea28d93047 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:46.848834 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330224c8-d1d1-4715-a02b-ef26328b622f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:46.849484 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330224c8-d1d1-4715-a02b-ef26328b622f admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:46.849766 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330224c8-d1d1-4715-a02b-ef26328b622f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:46.850041 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330224c8-d1d1-4715-a02b-ef26328b622f admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:46.850763 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330224c8-d1d1-4715-a02b-ef26328b622f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:46.852386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330224c8-d1d1-4715-a02b-ef26328b622f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:46.864615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1074/2134] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:46.868880 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071439b6-aecd-4b10-8b31-d43d04b9b6a6 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:46.882414 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071439b6-aecd-4b10-8b31-d43d04b9b6a6 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:46.885766 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f23772-ff1d-4867-889d-c837fef9cf6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:46.886766 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f23772-ff1d-4867-889d-c837fef9cf6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:46.887053 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f23772-ff1d-4867-889d-c837fef9cf6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:46.887329 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f23772-ff1d-4867-889d-c837fef9cf6f admin admin] PATH_INFO: `/v3/projects/8b0ad66a72b849b5a1a3170f871bf176/users/6fa63c4766b84898915ba5c79899c0e6/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:46.914278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f23772-ff1d-4867-889d-c837fef9cf6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b0ad66a72b849b5a1a3170f871bf176, user_id=6fa63c4766b84898915ba5c79899c0e6, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:46.917064 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f23772-ff1d-4867-889d-c837fef9cf6f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:46.920926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea84eb-095c-46fa-b336-c4842decdd4b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:46.922307 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea84eb-095c-46fa-b336-c4842decdd4b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:46.922307 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea84eb-095c-46fa-b336-c4842decdd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:46.945238 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-55ea84eb-095c-46fa-b336-c4842decdd4b None None] MFA Rules not processed for user `a6a6e5f6e5f24ba89ca4895aebe158f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:46.946882 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-55ea84eb-095c-46fa-b336-c4842decdd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:47.004697 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1075/2135] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:47.039895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1061/2136] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:08:46 2022] PUT /identity/v3/projects/8b0ad66a72b849b5a1a3170f871bf176/users/6fa63c4766b84898915ba5c79899c0e6/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:47.051982 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65a811-c7fa-43b1-b3c1-8576f9b4f296 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:47.052341 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65a811-c7fa-43b1-b3c1-8576f9b4f296 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:47.052721 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b65a811-c7fa-43b1-b3c1-8576f9b4f296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:47.086771 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9b65a811-c7fa-43b1-b3c1-8576f9b4f296 None None] MFA Rules not processed for user `6fa63c4766b84898915ba5c79899c0e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:47.088334 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9b65a811-c7fa-43b1-b3c1-8576f9b4f296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:47.157805 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1076/2137] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:47.169307 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f23772-ff1d-4867-889d-c837fef9cf6f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:47.171087 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f7f23772-ff1d-4867-889d-c837fef9cf6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:47.241831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f23772-ff1d-4867-889d-c837fef9cf6f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:47.245600 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9e55d-60dc-4f07-8c0a-76ec304d9d3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:47.246256 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9e55d-60dc-4f07-8c0a-76ec304d9d3e service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:47.246537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9e55d-60dc-4f07-8c0a-76ec304d9d3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:47.246811 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9e55d-60dc-4f07-8c0a-76ec304d9d3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:47.257141 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f9e55d-60dc-4f07-8c0a-76ec304d9d3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:47.258769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f9e55d-60dc-4f07-8c0a-76ec304d9d3e service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:47.269450 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1062/2138] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:08:47.433451 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:49.256201 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:51.976169 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65a811-c7fa-43b1-b3c1-8576f9b4f296 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:52.045110 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b65a811-c7fa-43b1-b3c1-8576f9b4f296 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:52.050175 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ba6c6-608b-4956-b541-c420e7edf74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:52.051076 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ba6c6-608b-4956-b541-c420e7edf74d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:52.051360 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ba6c6-608b-4956-b541-c420e7edf74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:52.051635 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ba6c6-608b-4956-b541-c420e7edf74d admin admin] PATH_INFO: `/v3/users/6fa63c4766b84898915ba5c79899c0e6` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:52.052859 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ba6c6-608b-4956-b541-c420e7edf74d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fa63c4766b84898915ba5c79899c0e6)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:52.054853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ba6c6-608b-4956-b541-c420e7edf74d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:52.322492 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-801ba6c6-608b-4956-b541-c420e7edf74d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fa63c4766b84898915ba5c79899c0e6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:52.323336 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-801ba6c6-608b-4956-b541-c420e7edf74d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fa63c4766b84898915ba5c79899c0e6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:52.410186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-801ba6c6-608b-4956-b541-c420e7edf74d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fa63c4766b84898915ba5c79899c0e6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:52.414430 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-801ba6c6-608b-4956-b541-c420e7edf74d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fa63c4766b84898915ba5c79899c0e6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:52.433417 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-801ba6c6-608b-4956-b541-c420e7edf74d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fa63c4766b84898915ba5c79899c0e6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:52.460647 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1077/2139] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:08:51 2022] DELETE /identity/v3/users/6fa63c4766b84898915ba5c79899c0e6 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:52.462909 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee8f15-08e7-4fde-bce2-46de2498601c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:52.463367 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee8f15-08e7-4fde-bce2-46de2498601c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:52.463516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee8f15-08e7-4fde-bce2-46de2498601c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:52.491418 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-cdee8f15-08e7-4fde-bce2-46de2498601c None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:52.492933 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-cdee8f15-08e7-4fde-bce2-46de2498601c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:52.581484 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1063/2140] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:52.595649 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ba6c6-608b-4956-b541-c420e7edf74d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:52.598584 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-801ba6c6-608b-4956-b541-c420e7edf74d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:52.688804 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ba6c6-608b-4956-b541-c420e7edf74d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:52.693604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988aeac-dfea-4cd4-bb0b-545f0ae983cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:52.693997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988aeac-dfea-4cd4-bb0b-545f0ae983cc service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:52.694278 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988aeac-dfea-4cd4-bb0b-545f0ae983cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:52.694550 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988aeac-dfea-4cd4-bb0b-545f0ae983cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:52.705903 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f988aeac-dfea-4cd4-bb0b-545f0ae983cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:52.707428 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f988aeac-dfea-4cd4-bb0b-545f0ae983cc service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:52.712813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:52.718063 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1078/2141] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:08:53.274814 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee8f15-08e7-4fde-bce2-46de2498601c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:53.276462 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-cdee8f15-08e7-4fde-bce2-46de2498601c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:53.290195 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee8f15-08e7-4fde-bce2-46de2498601c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:53.296592 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a869e17-8eff-4b2c-bb9e-8722d88d7c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:53.296592 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a869e17-8eff-4b2c-bb9e-8722d88d7c5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:53.296592 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a869e17-8eff-4b2c-bb9e-8722d88d7c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:53.297132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a869e17-8eff-4b2c-bb9e-8722d88d7c5c admin admin] PATH_INFO: `/v3/projects/8b0ad66a72b849b5a1a3170f871bf176` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:53.298586 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a869e17-8eff-4b2c-bb9e-8722d88d7c5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b0ad66a72b849b5a1a3170f871bf176)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:53.301195 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a869e17-8eff-4b2c-bb9e-8722d88d7c5c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:53.431595 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:53.445715 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3a869e17-8eff-4b2c-bb9e-8722d88d7c5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b0ad66a72b849b5a1a3170f871bf176', 'request_id': 'req-3a869e17-8eff-4b2c-bb9e-8722d88d7c5c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:53.468146 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3a869e17-8eff-4b2c-bb9e-8722d88d7c5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b0ad66a72b849b5a1a3170f871bf176', 'request_id': 'req-3a869e17-8eff-4b2c-bb9e-8722d88d7c5c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:53.484494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3a869e17-8eff-4b2c-bb9e-8722d88d7c5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b0ad66a72b849b5a1a3170f871bf176', 'request_id': 'req-3a869e17-8eff-4b2c-bb9e-8722d88d7c5c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:53.720480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d3fa0-8041-4c48-bf49-cb847fa82c0a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:53.720937 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d3fa0-8041-4c48-bf49-cb847fa82c0a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:53.721119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d3fa0-8041-4c48-bf49-cb847fa82c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:53.730362 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3a869e17-8eff-4b2c-bb9e-8722d88d7c5c admin admin] The token cache is being invalidate because project 8b0ad66a72b849b5a1a3170f871bf176 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:08:53.730653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3a869e17-8eff-4b2c-bb9e-8722d88d7c5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:53.733156 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3a869e17-8eff-4b2c-bb9e-8722d88d7c5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:53.741499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1064/2142] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:08:53 2022] DELETE /identity/v3/projects/8b0ad66a72b849b5a1a3170f871bf176 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:53.746201 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65083570-964c-40d7-b1ad-eb1c46cb8dac None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:53.746706 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65083570-964c-40d7-b1ad-eb1c46cb8dac None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:53.747164 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65083570-964c-40d7-b1ad-eb1c46cb8dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:53.749249 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d33d3fa0-8041-4c48-bf49-cb847fa82c0a None None] MFA Rules not processed for user `f041449191c94a5ca5a38c29dd146a15`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:53.752076 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d33d3fa0-8041-4c48-bf49-cb847fa82c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:53.780594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-65083570-964c-40d7-b1ad-eb1c46cb8dac None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:53.782580 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-65083570-964c-40d7-b1ad-eb1c46cb8dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:53.837250 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1079/2143] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:08:53.869395 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1065/2144] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:53.878944 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fffcce-10ec-441c-a66d-c025980253f7 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:53.879239 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fffcce-10ec-441c-a66d-c025980253f7 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:53.879616 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fffcce-10ec-441c-a66d-c025980253f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:53.900523 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-77fffcce-10ec-441c-a66d-c025980253f7 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:53.901849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-77fffcce-10ec-441c-a66d-c025980253f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:53.920196 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1080/2145] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:53.925422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65083570-964c-40d7-b1ad-eb1c46cb8dac None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:53.947071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65083570-964c-40d7-b1ad-eb1c46cb8dac None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:53.952994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d7f67-4e17-401d-b21d-104deda8290b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:53.954876 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d7f67-4e17-401d-b21d-104deda8290b admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:53.954989 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d7f67-4e17-401d-b21d-104deda8290b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:53.955220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d7f67-4e17-401d-b21d-104deda8290b admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:53.955930 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d7f67-4e17-401d-b21d-104deda8290b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:08:53.956450 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d7f67-4e17-401d-b21d-104deda8290b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:53.958827 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d7f67-4e17-401d-b21d-104deda8290b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:53.973150 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1066/2146] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:08:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:08:53.977887 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fffcce-10ec-441c-a66d-c025980253f7 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:53.994284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fffcce-10ec-441c-a66d-c025980253f7 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:54.002330 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c35f6-965c-4ccb-9566-d0baaa095579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:54.003447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c35f6-965c-4ccb-9566-d0baaa095579 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:54.003560 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c35f6-965c-4ccb-9566-d0baaa095579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:54.003853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c35f6-965c-4ccb-9566-d0baaa095579 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:54.004604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c35f6-965c-4ccb-9566-d0baaa095579 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:54.007743 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c35f6-965c-4ccb-9566-d0baaa095579 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:54.184699 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1081/2147] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:08:53 2022] POST /identity/v3/projects => generated 375 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:54.188922 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d7f67-4e17-401d-b21d-104deda8290b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:54.199154 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d7f67-4e17-401d-b21d-104deda8290b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:54.256352 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc540ccd-97cc-41ac-8c3b-cde89ca8bdbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:54.257025 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc540ccd-97cc-41ac-8c3b-cde89ca8bdbc admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:54.257313 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc540ccd-97cc-41ac-8c3b-cde89ca8bdbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:54.257597 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc540ccd-97cc-41ac-8c3b-cde89ca8bdbc admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:54.258304 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc540ccd-97cc-41ac-8c3b-cde89ca8bdbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:54.260426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc540ccd-97cc-41ac-8c3b-cde89ca8bdbc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:54.446105 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1067/2148] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:08:54 2022] POST /identity/v3/users => generated 288 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:08:54.450845 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c35f6-965c-4ccb-9566-d0baaa095579 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:54.461790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c35f6-965c-4ccb-9566-d0baaa095579 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:54.466613 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee64bf1-0153-445a-872c-297dc36eec00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:54.467351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee64bf1-0153-445a-872c-297dc36eec00 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:54.467700 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee64bf1-0153-445a-872c-297dc36eec00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:54.468038 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee64bf1-0153-445a-872c-297dc36eec00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:54.468897 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee64bf1-0153-445a-872c-297dc36eec00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:54.471228 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee64bf1-0153-445a-872c-297dc36eec00 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:54.485170 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1082/2149] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:54.486424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc540ccd-97cc-41ac-8c3b-cde89ca8bdbc admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:54.496839 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc540ccd-97cc-41ac-8c3b-cde89ca8bdbc admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:54.501721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09cb1c-6279-4cbf-ad83-477f30b38ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:54.509479 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09cb1c-6279-4cbf-ad83-477f30b38ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:54.509479 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09cb1c-6279-4cbf-ad83-477f30b38ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:54.509479 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09cb1c-6279-4cbf-ad83-477f30b38ac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:54.509479 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09cb1c-6279-4cbf-ad83-477f30b38ac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:54.509479 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09cb1c-6279-4cbf-ad83-477f30b38ac9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:54.515753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1068/2150] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:54.520732 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee64bf1-0153-445a-872c-297dc36eec00 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:54.530859 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee64bf1-0153-445a-872c-297dc36eec00 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:54.540128 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb2c2b-5407-4877-b14d-780fe4debd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:54.541050 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb2c2b-5407-4877-b14d-780fe4debd1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:54.541332 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb2c2b-5407-4877-b14d-780fe4debd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:54.541606 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb2c2b-5407-4877-b14d-780fe4debd1d admin admin] PATH_INFO: `/v3/projects/af7fdd651c2b4c8a8452e7b1d0889470/users/13cda543e1e5467983687551273c0e71/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:54.566418 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eb2c2b-5407-4877-b14d-780fe4debd1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=af7fdd651c2b4c8a8452e7b1d0889470, user_id=13cda543e1e5467983687551273c0e71, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:54.566418 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eb2c2b-5407-4877-b14d-780fe4debd1d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:54.668060 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1083/2151] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:08:54 2022] PUT /identity/v3/projects/af7fdd651c2b4c8a8452e7b1d0889470/users/13cda543e1e5467983687551273c0e71/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:54.678065 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93c1b6-f987-4d34-916c-a9bcb307013a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:54.678365 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93c1b6-f987-4d34-916c-a9bcb307013a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:54.678645 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93c1b6-f987-4d34-916c-a9bcb307013a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:54.706267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7a93c1b6-f987-4d34-916c-a9bcb307013a None None] MFA Rules not processed for user `13cda543e1e5467983687551273c0e71`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:54.707541 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7a93c1b6-f987-4d34-916c-a9bcb307013a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:54.776257 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1069/2152] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:54.787605 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb2c2b-5407-4877-b14d-780fe4debd1d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:54.789364 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-08eb2c2b-5407-4877-b14d-780fe4debd1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:54.869752 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb2c2b-5407-4877-b14d-780fe4debd1d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:54.873775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d9512-2639-4baa-9976-5a1d550fb37a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:54.874700 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d9512-2639-4baa-9976-5a1d550fb37a service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:54.874700 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d9512-2639-4baa-9976-5a1d550fb37a service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:54.874971 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d9512-2639-4baa-9976-5a1d550fb37a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:54.893308 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d9512-2639-4baa-9976-5a1d550fb37a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:54.894514 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d9512-2639-4baa-9976-5a1d550fb37a service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:54.905408 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1084/2153] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:08:55.872617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93c1b6-f987-4d34-916c-a9bcb307013a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:55.874004 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7a93c1b6-f987-4d34-916c-a9bcb307013a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:55.943136 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93c1b6-f987-4d34-916c-a9bcb307013a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:55.948373 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da873de2-8236-4112-b8a3-7f9cdab15f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:55.949239 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da873de2-8236-4112-b8a3-7f9cdab15f99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:55.949517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da873de2-8236-4112-b8a3-7f9cdab15f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:55.949790 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da873de2-8236-4112-b8a3-7f9cdab15f99 admin admin] PATH_INFO: `/v3/users/d26da25b32cd434795aa19ec60e031bc` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:55.950969 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da873de2-8236-4112-b8a3-7f9cdab15f99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d26da25b32cd434795aa19ec60e031bc)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:55.962530 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:55.966338 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da873de2-8236-4112-b8a3-7f9cdab15f99 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:56.253392 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-da873de2-8236-4112-b8a3-7f9cdab15f99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd26da25b32cd434795aa19ec60e031bc'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:56.254047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-da873de2-8236-4112-b8a3-7f9cdab15f99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd26da25b32cd434795aa19ec60e031bc'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:56.266741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-da873de2-8236-4112-b8a3-7f9cdab15f99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd26da25b32cd434795aa19ec60e031bc'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:56.273799 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-da873de2-8236-4112-b8a3-7f9cdab15f99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd26da25b32cd434795aa19ec60e031bc'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:56.409723 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-da873de2-8236-4112-b8a3-7f9cdab15f99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd26da25b32cd434795aa19ec60e031bc'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:56.417992 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1070/2154] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:08:55 2022] DELETE /identity/v3/users/d26da25b32cd434795aa19ec60e031bc => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:56.425880 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f680f7-8d50-4f8c-b152-fe71298e7164 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:56.427289 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f680f7-8d50-4f8c-b152-fe71298e7164 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:56.427771 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f680f7-8d50-4f8c-b152-fe71298e7164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:56.447649 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-98f680f7-8d50-4f8c-b152-fe71298e7164 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:56.449248 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-98f680f7-8d50-4f8c-b152-fe71298e7164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:56.494632 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:56.514289 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1085/2155] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:56.524078 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da873de2-8236-4112-b8a3-7f9cdab15f99 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:56.525627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-da873de2-8236-4112-b8a3-7f9cdab15f99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:56.583442 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da873de2-8236-4112-b8a3-7f9cdab15f99 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:56.587216 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f0a68-9e83-4ff2-b4df-2afa79d9f286 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:56.587863 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f0a68-9e83-4ff2-b4df-2afa79d9f286 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:56.588149 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f0a68-9e83-4ff2-b4df-2afa79d9f286 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:56.588457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f0a68-9e83-4ff2-b4df-2afa79d9f286 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:56.597766 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f0a68-9e83-4ff2-b4df-2afa79d9f286 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:56.599239 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f0a68-9e83-4ff2-b4df-2afa79d9f286 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:56.608863 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1071/2156] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:08:57.121702 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f680f7-8d50-4f8c-b152-fe71298e7164 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:57.123389 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-98f680f7-8d50-4f8c-b152-fe71298e7164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:57.158020 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f680f7-8d50-4f8c-b152-fe71298e7164 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:57.162362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1f5175-96b4-44bd-b95a-909261ad0bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:57.163230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f5175-96b4-44bd-b95a-909261ad0bb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:57.164094 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f5175-96b4-44bd-b95a-909261ad0bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:57.164452 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f5175-96b4-44bd-b95a-909261ad0bb7 admin admin] PATH_INFO: `/v3/projects/06fdc22c735647dd802b7d19b590fe30` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:57.165893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f5175-96b4-44bd-b95a-909261ad0bb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06fdc22c735647dd802b7d19b590fe30)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:57.167816 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1f5175-96b4-44bd-b95a-909261ad0bb7 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:57.299677 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6d1f5175-96b4-44bd-b95a-909261ad0bb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06fdc22c735647dd802b7d19b590fe30', 'request_id': 'req-6d1f5175-96b4-44bd-b95a-909261ad0bb7'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:57.433377 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6d1f5175-96b4-44bd-b95a-909261ad0bb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06fdc22c735647dd802b7d19b590fe30', 'request_id': 'req-6d1f5175-96b4-44bd-b95a-909261ad0bb7'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:57.441741 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6d1f5175-96b4-44bd-b95a-909261ad0bb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06fdc22c735647dd802b7d19b590fe30', 'request_id': 'req-6d1f5175-96b4-44bd-b95a-909261ad0bb7'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:57.457968 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6d1f5175-96b4-44bd-b95a-909261ad0bb7 admin admin] The token cache is being invalidate because project 06fdc22c735647dd802b7d19b590fe30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:08:57.458385 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6d1f5175-96b4-44bd-b95a-909261ad0bb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:57.459258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6d1f5175-96b4-44bd-b95a-909261ad0bb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:57.461043 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1086/2157] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:08:57 2022] DELETE /identity/v3/projects/06fdc22c735647dd802b7d19b590fe30 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:57.465911 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56e65e-7927-484c-a9e4-4e916f31211f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:57.466353 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56e65e-7927-484c-a9e4-4e916f31211f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:57.466459 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56e65e-7927-484c-a9e4-4e916f31211f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:57.483924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-0c56e65e-7927-484c-a9e4-4e916f31211f None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:57.485163 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0c56e65e-7927-484c-a9e4-4e916f31211f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:57.542978 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1072/2158] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:08:57.552436 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d0403-328b-4f18-9da6-29f39d3bd587 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:57.552694 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d0403-328b-4f18-9da6-29f39d3bd587 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:57.552973 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d0403-328b-4f18-9da6-29f39d3bd587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:57.572861 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f68d0403-328b-4f18-9da6-29f39d3bd587 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:57.574186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f68d0403-328b-4f18-9da6-29f39d3bd587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:57.582767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1087/2159] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:57.587223 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56e65e-7927-484c-a9e4-4e916f31211f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:57.597618 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56e65e-7927-484c-a9e4-4e916f31211f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:57.602245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c85c94-3e5f-4732-96c1-ed1dec8a78e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:57.602963 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c85c94-3e5f-4732-96c1-ed1dec8a78e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:57.603238 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c85c94-3e5f-4732-96c1-ed1dec8a78e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:57.605801 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c85c94-3e5f-4732-96c1-ed1dec8a78e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:57.605801 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c85c94-3e5f-4732-96c1-ed1dec8a78e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:08:57.605801 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c85c94-3e5f-4732-96c1-ed1dec8a78e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:57.606271 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c85c94-3e5f-4732-96c1-ed1dec8a78e1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:57.618289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1073/2160] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:08:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:08:57.625783 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d0403-328b-4f18-9da6-29f39d3bd587 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:57.635537 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d0403-328b-4f18-9da6-29f39d3bd587 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:57.640118 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47569a2d-7a3f-4424-9334-bd6489fd31c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:57.640786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47569a2d-7a3f-4424-9334-bd6489fd31c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:57.641091 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47569a2d-7a3f-4424-9334-bd6489fd31c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:57.641360 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47569a2d-7a3f-4424-9334-bd6489fd31c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:57.642051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47569a2d-7a3f-4424-9334-bd6489fd31c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:57.644534 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47569a2d-7a3f-4424-9334-bd6489fd31c7 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:57.724442 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1088/2161] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:08:57 2022] POST /identity/v3/projects => generated 405 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:57.725550 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c85c94-3e5f-4732-96c1-ed1dec8a78e1 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:57.738627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c85c94-3e5f-4732-96c1-ed1dec8a78e1 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:57.789559 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47569a2d-7a3f-4424-9334-bd6489fd31c7 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:57.791461 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-47569a2d-7a3f-4424-9334-bd6489fd31c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:57.810278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e5aab-d6df-4adb-9c86-ae85159aa7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:57.811017 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e5aab-d6df-4adb-9c86-ae85159aa7d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:57.811305 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e5aab-d6df-4adb-9c86-ae85159aa7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:57.811578 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e5aab-d6df-4adb-9c86-ae85159aa7d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:57.812273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e5aab-d6df-4adb-9c86-ae85159aa7d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:57.816958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e5aab-d6df-4adb-9c86-ae85159aa7d5 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:57.892341 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1074/2162] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:08:57 2022] POST /identity/v3/users => generated 303 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:08:57.895881 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e5aab-d6df-4adb-9c86-ae85159aa7d5 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:57.897350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47569a2d-7a3f-4424-9334-bd6489fd31c7 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:57.903007 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d070e-224d-44cd-bf05-ffd01f2bc784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:57.903910 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d070e-224d-44cd-bf05-ffd01f2bc784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:57.904259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d070e-224d-44cd-bf05-ffd01f2bc784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:57.904554 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d070e-224d-44cd-bf05-ffd01f2bc784 admin admin] PATH_INFO: `/v3/users/013d57b8957840a7bfc98ff17597b776` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:57.906033 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d070e-224d-44cd-bf05-ffd01f2bc784 admin admin] RBAC: Authorizing `identity:delete_user(user_id=013d57b8957840a7bfc98ff17597b776)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:57.907431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e5aab-d6df-4adb-9c86-ae85159aa7d5 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:57.908697 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26d070e-224d-44cd-bf05-ffd01f2bc784 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:57.913050 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4c347-16d9-412a-83f0-9304efdadafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:57.920590 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4c347-16d9-412a-83f0-9304efdadafb admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:57.920590 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4c347-16d9-412a-83f0-9304efdadafb admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:57.920590 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4c347-16d9-412a-83f0-9304efdadafb admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:57.920590 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4c347-16d9-412a-83f0-9304efdadafb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:57.920590 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4c347-16d9-412a-83f0-9304efdadafb admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:57.928375 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1075/2163] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:57.932149 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4c347-16d9-412a-83f0-9304efdadafb admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:57.942507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4c347-16d9-412a-83f0-9304efdadafb admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:57.947640 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722613f-37ba-4eb4-98a8-26a6d2a81b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:57.948368 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722613f-37ba-4eb4-98a8-26a6d2a81b57 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:57.948664 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722613f-37ba-4eb4-98a8-26a6d2a81b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:57.948959 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722613f-37ba-4eb4-98a8-26a6d2a81b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:57.949843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3722613f-37ba-4eb4-98a8-26a6d2a81b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:57.951505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3722613f-37ba-4eb4-98a8-26a6d2a81b57 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:57.972921 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1076/2164] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:08:57.977223 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722613f-37ba-4eb4-98a8-26a6d2a81b57 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:57.988658 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722613f-37ba-4eb4-98a8-26a6d2a81b57 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:57.993725 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800c4d7b-d1b5-4980-8330-e97c012f54dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:57.994727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c4d7b-d1b5-4980-8330-e97c012f54dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:57.995008 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c4d7b-d1b5-4980-8330-e97c012f54dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:57.995279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800c4d7b-d1b5-4980-8330-e97c012f54dc admin admin] PATH_INFO: `/v3/projects/97c1aac2dd8c4c55aef9c0ba641838c1/users/1ce66264151f409e87ae98e96868af18/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:58.018117 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800c4d7b-d1b5-4980-8330-e97c012f54dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=97c1aac2dd8c4c55aef9c0ba641838c1, user_id=1ce66264151f409e87ae98e96868af18, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:58.021284 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800c4d7b-d1b5-4980-8330-e97c012f54dc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:58.160139 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1077/2165] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:08:57 2022] PUT /identity/v3/projects/97c1aac2dd8c4c55aef9c0ba641838c1/users/1ce66264151f409e87ae98e96868af18/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:08:58.168002 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f26d070e-224d-44cd-bf05-ffd01f2bc784 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '013d57b8957840a7bfc98ff17597b776'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:58.168758 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f26d070e-224d-44cd-bf05-ffd01f2bc784 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '013d57b8957840a7bfc98ff17597b776'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:58.171667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15528f60-efa7-4aa3-a718-eff904b9d108 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:58.171667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15528f60-efa7-4aa3-a718-eff904b9d108 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:58.171877 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15528f60-efa7-4aa3-a718-eff904b9d108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:58.200413 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-15528f60-efa7-4aa3-a718-eff904b9d108 None None] MFA Rules not processed for user `1ce66264151f409e87ae98e96868af18`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:58.201652 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-15528f60-efa7-4aa3-a718-eff904b9d108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:58.240097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:58.249664 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f26d070e-224d-44cd-bf05-ffd01f2bc784 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '013d57b8957840a7bfc98ff17597b776'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:58.250695 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f26d070e-224d-44cd-bf05-ffd01f2bc784 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '013d57b8957840a7bfc98ff17597b776'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:58.265487 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f26d070e-224d-44cd-bf05-ffd01f2bc784 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '013d57b8957840a7bfc98ff17597b776'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:58.275981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1078/2166] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:58 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:58.280404 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15528f60-efa7-4aa3-a718-eff904b9d108 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:58.285752 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-15528f60-efa7-4aa3-a718-eff904b9d108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:58.286010 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1089/2167] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:08:57 2022] DELETE /identity/v3/users/013d57b8957840a7bfc98ff17597b776 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:58.287009 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e22399-f098-4587-8d4d-3cd2d04f7ffd None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:58.287305 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e22399-f098-4587-8d4d-3cd2d04f7ffd None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:58.287589 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e22399-f098-4587-8d4d-3cd2d04f7ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:58.305732 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e5e22399-f098-4587-8d4d-3cd2d04f7ffd None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:58.307491 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e5e22399-f098-4587-8d4d-3cd2d04f7ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:58.343508 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:58.358950 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15528f60-efa7-4aa3-a718-eff904b9d108 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:58.371282 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1090/2168] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:58.380705 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e22399-f098-4587-8d4d-3cd2d04f7ffd None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:58.382033 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e5e22399-f098-4587-8d4d-3cd2d04f7ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:58.415154 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4167d-e687-4a4c-bc9e-683cc7152831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:58.416111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4167d-e687-4a4c-bc9e-683cc7152831 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:58.416437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4167d-e687-4a4c-bc9e-683cc7152831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:58.416708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4167d-e687-4a4c-bc9e-683cc7152831 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:58.418133 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4167d-e687-4a4c-bc9e-683cc7152831 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:58.420558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4167d-e687-4a4c-bc9e-683cc7152831 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:58.440874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e22399-f098-4587-8d4d-3cd2d04f7ffd None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:58.445827 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2896c24c-6bea-4a74-ab89-654b92ccae8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:58.447794 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896c24c-6bea-4a74-ab89-654b92ccae8a service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:58.448516 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896c24c-6bea-4a74-ab89-654b92ccae8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:58.448750 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2896c24c-6bea-4a74-ab89-654b92ccae8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:58.459876 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2896c24c-6bea-4a74-ab89-654b92ccae8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:58.461086 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2896c24c-6bea-4a74-ab89-654b92ccae8a service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:58.471774 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1091/2169] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 20:08:58.512071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1079/2170] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:08:58 2022] POST /identity/v3/projects => generated 403 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:08:58.513040 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2896c24c-6bea-4a74-ab89-654b92ccae8a service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:58.524960 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2896c24c-6bea-4a74-ab89-654b92ccae8a service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:58.595630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0373aa6d-7baa-4f61-89b0-83c2537f2fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:58.597955 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373aa6d-7baa-4f61-89b0-83c2537f2fac admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:58.598624 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373aa6d-7baa-4f61-89b0-83c2537f2fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:58.599044 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0373aa6d-7baa-4f61-89b0-83c2537f2fac admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:58.600029 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0373aa6d-7baa-4f61-89b0-83c2537f2fac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:58.602166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0373aa6d-7baa-4f61-89b0-83c2537f2fac admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:58.694303 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1092/2171] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:08:58 2022] POST /identity/v3/users => generated 308 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:08:58.698822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4167d-e687-4a4c-bc9e-683cc7152831 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:58.712591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4167d-e687-4a4c-bc9e-683cc7152831 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:58.717967 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa23ef8-9a49-4a76-a5af-8ea4cb533523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:58.718736 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa23ef8-9a49-4a76-a5af-8ea4cb533523 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:58.719764 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa23ef8-9a49-4a76-a5af-8ea4cb533523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:58.719764 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa23ef8-9a49-4a76-a5af-8ea4cb533523 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:58.720536 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa23ef8-9a49-4a76-a5af-8ea4cb533523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:58.722508 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa23ef8-9a49-4a76-a5af-8ea4cb533523 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:58.737369 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1080/2172] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:58.738818 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0373aa6d-7baa-4f61-89b0-83c2537f2fac admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:58.752174 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0373aa6d-7baa-4f61-89b0-83c2537f2fac admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:58.757759 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690023b-7614-4104-86c1-65b6d9d65fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:58.758651 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690023b-7614-4104-86c1-65b6d9d65fde admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:58.758934 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690023b-7614-4104-86c1-65b6d9d65fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:58.759240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690023b-7614-4104-86c1-65b6d9d65fde admin admin] PATH_INFO: `/v3/projects/c532534f4caf44d2acc656f00827a80d/users/5195249ce4fe470e82df043da48aa041/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:58.775546 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5690023b-7614-4104-86c1-65b6d9d65fde admin admin] RBAC: Authorizing `identity:create_grant(project_id=c532534f4caf44d2acc656f00827a80d, user_id=5195249ce4fe470e82df043da48aa041, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:58.778277 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5690023b-7614-4104-86c1-65b6d9d65fde admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:58.891021 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1093/2173] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:08:58 2022] PUT /identity/v3/projects/c532534f4caf44d2acc656f00827a80d/users/5195249ce4fe470e82df043da48aa041/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:08:58.894749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa23ef8-9a49-4a76-a5af-8ea4cb533523 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:58.916241 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa23ef8-9a49-4a76-a5af-8ea4cb533523 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:59.083347 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328dc7be-1136-4606-9d94-85890e306296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:59.084221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328dc7be-1136-4606-9d94-85890e306296 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:59.084542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328dc7be-1136-4606-9d94-85890e306296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:59.084985 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328dc7be-1136-4606-9d94-85890e306296 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:59.086045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328dc7be-1136-4606-9d94-85890e306296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:59.088101 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328dc7be-1136-4606-9d94-85890e306296 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:59.103343 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1081/2174] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:08:59.105595 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690023b-7614-4104-86c1-65b6d9d65fde admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:59.120901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690023b-7614-4104-86c1-65b6d9d65fde admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:59.145967 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328dc7be-1136-4606-9d94-85890e306296 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:59.154875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-328dc7be-1136-4606-9d94-85890e306296 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:59.162403 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e6fff-c3cd-47bf-893f-5e0f6c57b8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:59.163339 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e6fff-c3cd-47bf-893f-5e0f6c57b8f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:59.163623 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e6fff-c3cd-47bf-893f-5e0f6c57b8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:59.167784 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e6fff-c3cd-47bf-893f-5e0f6c57b8f8 admin admin] PATH_INFO: `/v3/projects/c532534f4caf44d2acc656f00827a80d/users/5195249ce4fe470e82df043da48aa041/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:59.168265 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e6fff-c3cd-47bf-893f-5e0f6c57b8f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c532534f4caf44d2acc656f00827a80d, user_id=5195249ce4fe470e82df043da48aa041, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:59.171320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e6fff-c3cd-47bf-893f-5e0f6c57b8f8 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:59.178214 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328dc7be-1136-4606-9d94-85890e306296 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:59.197457 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1094/2175] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:08:59 2022] PUT /identity/v3/projects/c532534f4caf44d2acc656f00827a80d/users/5195249ce4fe470e82df043da48aa041/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:08:59.199139 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e20b2a-218c-46df-bfb3-08961808736f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:59.199486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e20b2a-218c-46df-bfb3-08961808736f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:59.199874 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e20b2a-218c-46df-bfb3-08961808736f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:59.235778 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-18e20b2a-218c-46df-bfb3-08961808736f None None] MFA Rules not processed for user `5195249ce4fe470e82df043da48aa041`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:59.238255 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-18e20b2a-218c-46df-bfb3-08961808736f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:59.314272 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537060f-0df3-474b-b118-3e733bdd68a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:59.315342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537060f-0df3-474b-b118-3e733bdd68a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:59.315972 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537060f-0df3-474b-b118-3e733bdd68a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:59.316418 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537060f-0df3-474b-b118-3e733bdd68a4 admin admin] PATH_INFO: `/v3/projects/4f03ef8c22ad4b58b93c6665bb2f5bba` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:59.318224 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537060f-0df3-474b-b118-3e733bdd68a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f03ef8c22ad4b58b93c6665bb2f5bba)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:59.320477 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537060f-0df3-474b-b118-3e733bdd68a4 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:59.366350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1095/2176] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:59 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:59.380199 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e20b2a-218c-46df-bfb3-08961808736f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:59.381036 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-18e20b2a-218c-46df-bfb3-08961808736f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:59.440158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e20b2a-218c-46df-bfb3-08961808736f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:59.444263 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246f56c-4e71-460f-be84-67fa71d5ccf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:59.444978 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246f56c-4e71-460f-be84-67fa71d5ccf1 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:59.445223 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246f56c-4e71-460f-be84-67fa71d5ccf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:59.445496 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246f56c-4e71-460f-be84-67fa71d5ccf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:59.457027 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3246f56c-4e71-460f-be84-67fa71d5ccf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:59.458484 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3246f56c-4e71-460f-be84-67fa71d5ccf1 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:59.474470 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1096/2177] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:08:59 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:08:59.536827 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b537060f-0df3-474b-b118-3e733bdd68a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f03ef8c22ad4b58b93c6665bb2f5bba', 'request_id': 'req-b537060f-0df3-474b-b118-3e733bdd68a4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:59.566697 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b537060f-0df3-474b-b118-3e733bdd68a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f03ef8c22ad4b58b93c6665bb2f5bba', 'request_id': 'req-b537060f-0df3-474b-b118-3e733bdd68a4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:59.581933 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b537060f-0df3-474b-b118-3e733bdd68a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f03ef8c22ad4b58b93c6665bb2f5bba', 'request_id': 'req-b537060f-0df3-474b-b118-3e733bdd68a4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:59.662145 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246f56c-4e71-460f-be84-67fa71d5ccf1 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:59.673603 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246f56c-4e71-460f-be84-67fa71d5ccf1 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:59.695273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b537060f-0df3-474b-b118-3e733bdd68a4 admin admin] The token cache is being invalidate because project 4f03ef8c22ad4b58b93c6665bb2f5bba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:08:59.695799 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b537060f-0df3-474b-b118-3e733bdd68a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:59.697844 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b537060f-0df3-474b-b118-3e733bdd68a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:59.700435 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1082/2178] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:08:59 2022] DELETE /identity/v3/projects/4f03ef8c22ad4b58b93c6665bb2f5bba => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:08:59.705096 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c7714-97a9-4557-9f69-d44eb020dc02 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:59.705574 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c7714-97a9-4557-9f69-d44eb020dc02 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:59.706032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c7714-97a9-4557-9f69-d44eb020dc02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:59.730958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-010c7714-97a9-4557-9f69-d44eb020dc02 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:59.732334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-010c7714-97a9-4557-9f69-d44eb020dc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:59.772438 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616ff03-854f-49da-9fc7-19406861e9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:59.773369 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616ff03-854f-49da-9fc7-19406861e9aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:59.774424 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616ff03-854f-49da-9fc7-19406861e9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:59.774424 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616ff03-854f-49da-9fc7-19406861e9aa admin admin] PATH_INFO: `/v3/users/1ce66264151f409e87ae98e96868af18` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:59.775360 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b616ff03-854f-49da-9fc7-19406861e9aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ce66264151f409e87ae98e96868af18)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:59.777384 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b616ff03-854f-49da-9fc7-19406861e9aa admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:59.812333 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1083/2179] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:59.818740 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21edfd-0d54-43f5-9a21-a06ca075e843 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:59.819222 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21edfd-0d54-43f5-9a21-a06ca075e843 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:59.819743 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21edfd-0d54-43f5-9a21-a06ca075e843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:59.838932 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1f21edfd-0d54-43f5-9a21-a06ca075e843 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:08:59.840216 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1f21edfd-0d54-43f5-9a21-a06ca075e843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:08:59.848948 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1084/2180] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:08:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:08:59.853141 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21edfd-0d54-43f5-9a21-a06ca075e843 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:59.862316 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21edfd-0d54-43f5-9a21-a06ca075e843 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:59.867877 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c23f99-cde2-488c-a6ab-314d59996744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:59.868679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c23f99-cde2-488c-a6ab-314d59996744 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:59.869027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c23f99-cde2-488c-a6ab-314d59996744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:59.869366 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c23f99-cde2-488c-a6ab-314d59996744 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:59.870168 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c23f99-cde2-488c-a6ab-314d59996744 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:08:59.870536 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c23f99-cde2-488c-a6ab-314d59996744 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:59.872240 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c23f99-cde2-488c-a6ab-314d59996744 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:59.885096 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1085/2181] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 20:08:59.889135 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c23f99-cde2-488c-a6ab-314d59996744 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:08:59.901105 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c23f99-cde2-488c-a6ab-314d59996744 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:08:59.907678 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38800a76-eb81-4b8f-acce-7c75542dd1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:08:59.908173 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38800a76-eb81-4b8f-acce-7c75542dd1bd admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:08:59.908751 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38800a76-eb81-4b8f-acce-7c75542dd1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:08:59.909101 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38800a76-eb81-4b8f-acce-7c75542dd1bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:08:59.910838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38800a76-eb81-4b8f-acce-7c75542dd1bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:08:59.912860 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38800a76-eb81-4b8f-acce-7c75542dd1bd admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:08:59.977788 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:08:59.993447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b616ff03-854f-49da-9fc7-19406861e9aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ce66264151f409e87ae98e96868af18'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:08:59.994053 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b616ff03-854f-49da-9fc7-19406861e9aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ce66264151f409e87ae98e96868af18'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:00.023068 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1086/2182] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:08:59 2022] POST /identity/v3/projects => generated 377 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:09:00.027178 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38800a76-eb81-4b8f-acce-7c75542dd1bd admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:00.072729 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b616ff03-854f-49da-9fc7-19406861e9aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ce66264151f409e87ae98e96868af18'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:00.073365 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b616ff03-854f-49da-9fc7-19406861e9aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ce66264151f409e87ae98e96868af18'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:00.091262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b616ff03-854f-49da-9fc7-19406861e9aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ce66264151f409e87ae98e96868af18'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:00.115052 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1097/2183] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:08:59 2022] DELETE /identity/v3/users/1ce66264151f409e87ae98e96868af18 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:00.123008 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38800a76-eb81-4b8f-acce-7c75542dd1bd admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:00.124686 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2ccc3-f101-4f9f-83dc-860f1537ca40 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:00.125025 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2ccc3-f101-4f9f-83dc-860f1537ca40 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:00.125252 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2ccc3-f101-4f9f-83dc-860f1537ca40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:00.146701 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-6bd2ccc3-f101-4f9f-83dc-860f1537ca40 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:00.147948 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6bd2ccc3-f101-4f9f-83dc-860f1537ca40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:00.215597 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39ffaf-4e59-4403-801f-38ddd0e3f59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:00.217715 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39ffaf-4e59-4403-801f-38ddd0e3f59a admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:00.218080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39ffaf-4e59-4403-801f-38ddd0e3f59a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:00.218424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a39ffaf-4e59-4403-801f-38ddd0e3f59a admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:00.219208 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a39ffaf-4e59-4403-801f-38ddd0e3f59a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:00.222763 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a39ffaf-4e59-4403-801f-38ddd0e3f59a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:00.229983 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:00.261805 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1098/2184] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:00.273365 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2ccc3-f101-4f9f-83dc-860f1537ca40 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:00.275138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6bd2ccc3-f101-4f9f-83dc-860f1537ca40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:00.319353 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1087/2185] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:09:00 2022] POST /identity/v3/users => generated 289 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:09:00.322862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39ffaf-4e59-4403-801f-38ddd0e3f59a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:00.325838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8a39ffaf-4e59-4403-801f-38ddd0e3f59a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:00.342483 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a39ffaf-4e59-4403-801f-38ddd0e3f59a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:00.353810 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7abda-8a38-49e5-bfaf-1220d0ae82fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:00.355796 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7abda-8a38-49e5-bfaf-1220d0ae82fb admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:00.356404 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7abda-8a38-49e5-bfaf-1220d0ae82fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:00.356800 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7abda-8a38-49e5-bfaf-1220d0ae82fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:00.357226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2ccc3-f101-4f9f-83dc-860f1537ca40 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:00.360829 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a7abda-8a38-49e5-bfaf-1220d0ae82fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:00.360829 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a7abda-8a38-49e5-bfaf-1220d0ae82fb admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:00.361760 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567eccb1-c903-46e9-bd69-f0acc1cc48f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:00.363025 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567eccb1-c903-46e9-bd69-f0acc1cc48f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:00.363417 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567eccb1-c903-46e9-bd69-f0acc1cc48f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:00.363780 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567eccb1-c903-46e9-bd69-f0acc1cc48f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:00.378041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1088/2186] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:09:00.379727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7abda-8a38-49e5-bfaf-1220d0ae82fb admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:00.386928 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567eccb1-c903-46e9-bd69-f0acc1cc48f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:00.388522 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567eccb1-c903-46e9-bd69-f0acc1cc48f5 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:00.391302 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7abda-8a38-49e5-bfaf-1220d0ae82fb admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:00.402475 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facffaa6-877f-4cd1-aa62-9f0323ca9edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:00.403594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facffaa6-877f-4cd1-aa62-9f0323ca9edf admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:00.403941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facffaa6-877f-4cd1-aa62-9f0323ca9edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:00.404283 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facffaa6-877f-4cd1-aa62-9f0323ca9edf admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:00.405149 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facffaa6-877f-4cd1-aa62-9f0323ca9edf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:00.407244 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facffaa6-877f-4cd1-aa62-9f0323ca9edf admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:00.424260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1089/2187] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:09:00.426193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facffaa6-877f-4cd1-aa62-9f0323ca9edf admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:00.431897 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1099/2188] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:09:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 20:09:00.454096 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facffaa6-877f-4cd1-aa62-9f0323ca9edf admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:00.469569 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c88155-0f7b-43cc-8969-8a51df9f701f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:00.470870 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c88155-0f7b-43cc-8969-8a51df9f701f admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:00.471161 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c88155-0f7b-43cc-8969-8a51df9f701f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:00.471485 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c88155-0f7b-43cc-8969-8a51df9f701f admin admin] PATH_INFO: `/v3/projects/6563af64e83446be985451f9b66bb112/users/68f69257b4c94da4b7370e6a5cbfe544/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:00.493234 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c88155-0f7b-43cc-8969-8a51df9f701f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6563af64e83446be985451f9b66bb112, user_id=68f69257b4c94da4b7370e6a5cbfe544, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:00.496244 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c88155-0f7b-43cc-8969-8a51df9f701f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:00.593193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1090/2189] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:09:00 2022] PUT /identity/v3/projects/6563af64e83446be985451f9b66bb112/users/68f69257b4c94da4b7370e6a5cbfe544/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:00.604845 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da73ac-0cf2-4ec9-9778-e2894f8ce22a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:00.605244 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da73ac-0cf2-4ec9-9778-e2894f8ce22a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:00.605615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da73ac-0cf2-4ec9-9778-e2894f8ce22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:00.638909 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-89da73ac-0cf2-4ec9-9778-e2894f8ce22a None None] MFA Rules not processed for user `68f69257b4c94da4b7370e6a5cbfe544`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:00.641131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-89da73ac-0cf2-4ec9-9778-e2894f8ce22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:00.913438 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1100/2190] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:00.925423 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c88155-0f7b-43cc-8969-8a51df9f701f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:00.933111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-47c88155-0f7b-43cc-8969-8a51df9f701f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:01.111114 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c88155-0f7b-43cc-8969-8a51df9f701f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:01.127763 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6240c5-e239-4de2-a6ea-9fe9a670979b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:01.129348 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6240c5-e239-4de2-a6ea-9fe9a670979b service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:01.129961 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6240c5-e239-4de2-a6ea-9fe9a670979b service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:01.130320 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6240c5-e239-4de2-a6ea-9fe9a670979b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:01.152747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6240c5-e239-4de2-a6ea-9fe9a670979b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:01.154148 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6240c5-e239-4de2-a6ea-9fe9a670979b service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:01.177823 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da73ac-0cf2-4ec9-9778-e2894f8ce22a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:01.179919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-89da73ac-0cf2-4ec9-9778-e2894f8ce22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:01.181576 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1091/2191] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:09:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:09:01.399643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da73ac-0cf2-4ec9-9778-e2894f8ce22a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:01.405465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63977b51-cb72-4bb9-b27b-e589b7f3909f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:01.408543 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63977b51-cb72-4bb9-b27b-e589b7f3909f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:01.409001 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63977b51-cb72-4bb9-b27b-e589b7f3909f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:01.409931 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63977b51-cb72-4bb9-b27b-e589b7f3909f admin admin] PATH_INFO: `/v3/projects/97c1aac2dd8c4c55aef9c0ba641838c1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:01.412035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63977b51-cb72-4bb9-b27b-e589b7f3909f admin admin] RBAC: Authorizing `identity:delete_project(project_id=97c1aac2dd8c4c55aef9c0ba641838c1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:01.415266 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63977b51-cb72-4bb9-b27b-e589b7f3909f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:01.438716 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:01.630811 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-63977b51-cb72-4bb9-b27b-e589b7f3909f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97c1aac2dd8c4c55aef9c0ba641838c1', 'request_id': 'req-63977b51-cb72-4bb9-b27b-e589b7f3909f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:01.833637 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-63977b51-cb72-4bb9-b27b-e589b7f3909f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97c1aac2dd8c4c55aef9c0ba641838c1', 'request_id': 'req-63977b51-cb72-4bb9-b27b-e589b7f3909f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:01.842035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-63977b51-cb72-4bb9-b27b-e589b7f3909f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97c1aac2dd8c4c55aef9c0ba641838c1', 'request_id': 'req-63977b51-cb72-4bb9-b27b-e589b7f3909f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:01.857866 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-63977b51-cb72-4bb9-b27b-e589b7f3909f admin admin] The token cache is being invalidate because project 97c1aac2dd8c4c55aef9c0ba641838c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:09:01.858367 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-63977b51-cb72-4bb9-b27b-e589b7f3909f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:01.860148 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-63977b51-cb72-4bb9-b27b-e589b7f3909f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:01.863950 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1101/2192] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:09:01 2022] DELETE /identity/v3/projects/97c1aac2dd8c4c55aef9c0ba641838c1 => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:01.867936 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6240c5-e239-4de2-a6ea-9fe9a670979b service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:01.900135 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6240c5-e239-4de2-a6ea-9fe9a670979b service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:01.955424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364262a9-c1df-4ece-a09c-c29ed71fd9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:01.956347 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364262a9-c1df-4ece-a09c-c29ed71fd9a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:01.956759 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364262a9-c1df-4ece-a09c-c29ed71fd9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:01.957049 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364262a9-c1df-4ece-a09c-c29ed71fd9a4 admin admin] PATH_INFO: `/v3/users/5195249ce4fe470e82df043da48aa041` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:01.958552 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364262a9-c1df-4ece-a09c-c29ed71fd9a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5195249ce4fe470e82df043da48aa041)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:01.960534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364262a9-c1df-4ece-a09c-c29ed71fd9a4 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:02.320443 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-364262a9-c1df-4ece-a09c-c29ed71fd9a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5195249ce4fe470e82df043da48aa041'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:02.320443 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-364262a9-c1df-4ece-a09c-c29ed71fd9a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5195249ce4fe470e82df043da48aa041'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:02.345429 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-364262a9-c1df-4ece-a09c-c29ed71fd9a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5195249ce4fe470e82df043da48aa041'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:02.369371 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-364262a9-c1df-4ece-a09c-c29ed71fd9a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5195249ce4fe470e82df043da48aa041'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:02.411442 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:02.433167 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-364262a9-c1df-4ece-a09c-c29ed71fd9a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5195249ce4fe470e82df043da48aa041'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:02.444811 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1092/2193] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:09:01 2022] DELETE /identity/v3/users/5195249ce4fe470e82df043da48aa041 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:03.016345 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:03.025315 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63977b51-cb72-4bb9-b27b-e589b7f3909f admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:03.026792 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-63977b51-cb72-4bb9-b27b-e589b7f3909f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:03.041457 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63977b51-cb72-4bb9-b27b-e589b7f3909f admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:03.137396 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbaa7a2-4393-478d-9d85-0ffdf77bc3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:03.141333 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbaa7a2-4393-478d-9d85-0ffdf77bc3ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:03.141666 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbaa7a2-4393-478d-9d85-0ffdf77bc3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:03.141947 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbaa7a2-4393-478d-9d85-0ffdf77bc3ee admin admin] PATH_INFO: `/v3/projects/c532534f4caf44d2acc656f00827a80d` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:03.147501 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbaa7a2-4393-478d-9d85-0ffdf77bc3ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=c532534f4caf44d2acc656f00827a80d)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:03.149752 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbaa7a2-4393-478d-9d85-0ffdf77bc3ee admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:03.403179 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dcbaa7a2-4393-478d-9d85-0ffdf77bc3ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c532534f4caf44d2acc656f00827a80d', 'request_id': 'req-dcbaa7a2-4393-478d-9d85-0ffdf77bc3ee'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:03.501606 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dcbaa7a2-4393-478d-9d85-0ffdf77bc3ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c532534f4caf44d2acc656f00827a80d', 'request_id': 'req-dcbaa7a2-4393-478d-9d85-0ffdf77bc3ee'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:03.512314 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dcbaa7a2-4393-478d-9d85-0ffdf77bc3ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c532534f4caf44d2acc656f00827a80d', 'request_id': 'req-dcbaa7a2-4393-478d-9d85-0ffdf77bc3ee'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:03.531067 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dcbaa7a2-4393-478d-9d85-0ffdf77bc3ee admin admin] The token cache is being invalidate because project c532534f4caf44d2acc656f00827a80d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:09:03.531737 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dcbaa7a2-4393-478d-9d85-0ffdf77bc3ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:03.532838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dcbaa7a2-4393-478d-9d85-0ffdf77bc3ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:03.534850 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1102/2194] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:09:03 2022] DELETE /identity/v3/projects/c532534f4caf44d2acc656f00827a80d => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:03.540053 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd22bd-114f-40e4-96e6-48af44e9385a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:03.540443 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd22bd-114f-40e4-96e6-48af44e9385a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:03.540745 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd22bd-114f-40e4-96e6-48af44e9385a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:03.560655 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-89fd22bd-114f-40e4-96e6-48af44e9385a None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:03.561887 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-89fd22bd-114f-40e4-96e6-48af44e9385a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:03.630443 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1093/2195] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:03.640105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9756b4a-a2e1-4666-9cbe-c6fe51694cde None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:03.640533 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9756b4a-a2e1-4666-9cbe-c6fe51694cde None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:03.640787 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9756b4a-a2e1-4666-9cbe-c6fe51694cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:03.669266 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d9756b4a-a2e1-4666-9cbe-c6fe51694cde None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:03.670507 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d9756b4a-a2e1-4666-9cbe-c6fe51694cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:03.678754 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1103/2196] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:03.683393 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd22bd-114f-40e4-96e6-48af44e9385a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:03.694365 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd22bd-114f-40e4-96e6-48af44e9385a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:03.700120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50569f3-6c91-4cf7-9c1b-8589c4ade98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:03.700843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50569f3-6c91-4cf7-9c1b-8589c4ade98b admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:03.701178 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50569f3-6c91-4cf7-9c1b-8589c4ade98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:03.701455 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50569f3-6c91-4cf7-9c1b-8589c4ade98b admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:03.702238 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50569f3-6c91-4cf7-9c1b-8589c4ade98b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:09:03.702549 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50569f3-6c91-4cf7-9c1b-8589c4ade98b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:03.704239 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50569f3-6c91-4cf7-9c1b-8589c4ade98b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:03.730423 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9756b4a-a2e1-4666-9cbe-c6fe51694cde None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:03.738420 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1094/2197] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:09:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:09:03.747495 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9756b4a-a2e1-4666-9cbe-c6fe51694cde None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:03.753903 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d19f4e-c713-4c25-ac8b-993ce3d061c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:03.754740 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d19f4e-c713-4c25-ac8b-993ce3d061c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:03.755056 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d19f4e-c713-4c25-ac8b-993ce3d061c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:03.755333 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d19f4e-c713-4c25-ac8b-993ce3d061c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:03.756044 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d19f4e-c713-4c25-ac8b-993ce3d061c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:03.759016 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d19f4e-c713-4c25-ac8b-993ce3d061c4 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:03.881416 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:03.896734 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1104/2198] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:09:03 2022] POST /identity/v3/projects => generated 365 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:09:03.898011 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50569f3-6c91-4cf7-9c1b-8589c4ade98b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:03.931684 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50569f3-6c91-4cf7-9c1b-8589c4ade98b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:04.037399 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc23b99-e351-4f7b-afa1-86b65fa338f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:04.042749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc23b99-e351-4f7b-afa1-86b65fa338f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:04.042749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc23b99-e351-4f7b-afa1-86b65fa338f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:04.042749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc23b99-e351-4f7b-afa1-86b65fa338f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:04.042749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc23b99-e351-4f7b-afa1-86b65fa338f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:04.042749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc23b99-e351-4f7b-afa1-86b65fa338f0 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:04.127535 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1095/2199] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:09:03 2022] POST /identity/v3/users => generated 283 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:09:04.129771 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d19f4e-c713-4c25-ac8b-993ce3d061c4 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:04.141056 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d19f4e-c713-4c25-ac8b-993ce3d061c4 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:04.145565 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290e989-9ab4-4fab-adb3-9a9a15c5a3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:04.146219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290e989-9ab4-4fab-adb3-9a9a15c5a3b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:04.146499 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290e989-9ab4-4fab-adb3-9a9a15c5a3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:04.146778 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8290e989-9ab4-4fab-adb3-9a9a15c5a3b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:04.147588 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8290e989-9ab4-4fab-adb3-9a9a15c5a3b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:04.149156 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8290e989-9ab4-4fab-adb3-9a9a15c5a3b5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:04.160574 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1105/2200] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:09:04.165074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc23b99-e351-4f7b-afa1-86b65fa338f0 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:04.176599 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc23b99-e351-4f7b-afa1-86b65fa338f0 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:04.186520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb045d7-03bf-4d04-a913-677bc9bc06a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:04.187608 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb045d7-03bf-4d04-a913-677bc9bc06a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:04.187901 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb045d7-03bf-4d04-a913-677bc9bc06a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:04.188869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb045d7-03bf-4d04-a913-677bc9bc06a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:04.189887 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb045d7-03bf-4d04-a913-677bc9bc06a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:04.192333 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb045d7-03bf-4d04-a913-677bc9bc06a4 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:04.204484 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1096/2201] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:09:04.209103 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290e989-9ab4-4fab-adb3-9a9a15c5a3b5 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:04.240517 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8290e989-9ab4-4fab-adb3-9a9a15c5a3b5 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:04.247026 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97957f-73a7-45e1-a3f9-505cf5626992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:04.247958 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97957f-73a7-45e1-a3f9-505cf5626992 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:04.248237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97957f-73a7-45e1-a3f9-505cf5626992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:04.248540 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97957f-73a7-45e1-a3f9-505cf5626992 admin admin] PATH_INFO: `/v3/projects/e608a3985b6e46bcafaebe462fa253be/users/5435ccbb0c954a96b7e393c509e9ea70/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:04.265664 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e97957f-73a7-45e1-a3f9-505cf5626992 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e608a3985b6e46bcafaebe462fa253be, user_id=5435ccbb0c954a96b7e393c509e9ea70, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:04.268429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e97957f-73a7-45e1-a3f9-505cf5626992 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:04.376953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1106/2202] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:09:04 2022] PUT /identity/v3/projects/e608a3985b6e46bcafaebe462fa253be/users/5435ccbb0c954a96b7e393c509e9ea70/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:04.387506 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e8253-80db-45cf-9215-ec40f5f20f3a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:04.387804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e8253-80db-45cf-9215-ec40f5f20f3a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:04.388087 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e8253-80db-45cf-9215-ec40f5f20f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:04.417543 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-753e8253-80db-45cf-9215-ec40f5f20f3a None None] MFA Rules not processed for user `5435ccbb0c954a96b7e393c509e9ea70`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:04.418808 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-753e8253-80db-45cf-9215-ec40f5f20f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:04.487177 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1097/2203] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:04 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:04.492239 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97957f-73a7-45e1-a3f9-505cf5626992 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:04.551732 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97957f-73a7-45e1-a3f9-505cf5626992 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:04.556415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a90a62-ec34-449b-a696-7297e67c22f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:04.557060 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a90a62-ec34-449b-a696-7297e67c22f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:04.557337 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a90a62-ec34-449b-a696-7297e67c22f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:04.557607 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a90a62-ec34-449b-a696-7297e67c22f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:04.558293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a90a62-ec34-449b-a696-7297e67c22f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:04.560195 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a90a62-ec34-449b-a696-7297e67c22f9 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:04.754878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1107/2204] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:09:04 2022] POST /identity/v3/projects => generated 365 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:09:04.758945 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e8253-80db-45cf-9215-ec40f5f20f3a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:04.770223 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e8253-80db-45cf-9215-ec40f5f20f3a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:04.826168 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ca849-96b2-408f-9426-5703b149dda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:04.827095 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ca849-96b2-408f-9426-5703b149dda0 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:04.827381 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ca849-96b2-408f-9426-5703b149dda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:04.827654 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ca849-96b2-408f-9426-5703b149dda0 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:04.828402 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5ca849-96b2-408f-9426-5703b149dda0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:04.830334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5ca849-96b2-408f-9426-5703b149dda0 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:04.948172 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1098/2205] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:09:04 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:09:04.952421 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a90a62-ec34-449b-a696-7297e67c22f9 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:04.962655 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a90a62-ec34-449b-a696-7297e67c22f9 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:04.967335 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b63f904-22c3-4171-9db5-86ce938c4b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:04.967987 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63f904-22c3-4171-9db5-86ce938c4b77 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:04.968324 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63f904-22c3-4171-9db5-86ce938c4b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:04.968609 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b63f904-22c3-4171-9db5-86ce938c4b77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:04.969358 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b63f904-22c3-4171-9db5-86ce938c4b77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:04.971237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b63f904-22c3-4171-9db5-86ce938c4b77 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:04.981662 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1108/2206] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:09:04.985895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ca849-96b2-408f-9426-5703b149dda0 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:04.996277 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ca849-96b2-408f-9426-5703b149dda0 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:05.000844 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d51bd-50b1-4c97-84f4-96065e9e82b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:05.001710 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d51bd-50b1-4c97-84f4-96065e9e82b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:05.001992 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d51bd-50b1-4c97-84f4-96065e9e82b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:05.002265 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59d51bd-50b1-4c97-84f4-96065e9e82b3 admin admin] PATH_INFO: `/v3/projects/24a782b8185d421d9796d6f67b54c624/users/ac10e6a7a7a14cad855f64d8a16b1016/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:05.017279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59d51bd-50b1-4c97-84f4-96065e9e82b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24a782b8185d421d9796d6f67b54c624, user_id=ac10e6a7a7a14cad855f64d8a16b1016, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:05.020117 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59d51bd-50b1-4c97-84f4-96065e9e82b3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:05.152926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1099/2207] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:09:04 2022] PUT /identity/v3/projects/24a782b8185d421d9796d6f67b54c624/users/ac10e6a7a7a14cad855f64d8a16b1016/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:05.157059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b63f904-22c3-4171-9db5-86ce938c4b77 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:05.167226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b63f904-22c3-4171-9db5-86ce938c4b77 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:05.223798 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07191556-a8d4-443b-96ac-d7963d90540f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:05.224490 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07191556-a8d4-443b-96ac-d7963d90540f admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:05.224776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07191556-a8d4-443b-96ac-d7963d90540f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:05.225051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07191556-a8d4-443b-96ac-d7963d90540f admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:05.225838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07191556-a8d4-443b-96ac-d7963d90540f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:05.227448 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07191556-a8d4-443b-96ac-d7963d90540f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:05.240748 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1109/2208] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:09:05.242518 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d51bd-50b1-4c97-84f4-96065e9e82b3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:05.251868 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:05.255706 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59d51bd-50b1-4c97-84f4-96065e9e82b3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:05.260837 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623206b-900c-4d2d-9638-22bed1e3617c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:05.261741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623206b-900c-4d2d-9638-22bed1e3617c admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:05.262021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623206b-900c-4d2d-9638-22bed1e3617c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:05.262327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623206b-900c-4d2d-9638-22bed1e3617c admin admin] PATH_INFO: `/v3/projects/24a782b8185d421d9796d6f67b54c624/users/ac10e6a7a7a14cad855f64d8a16b1016/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:05.264330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4623206b-900c-4d2d-9638-22bed1e3617c admin admin] RBAC: Authorizing `identity:create_grant(project_id=24a782b8185d421d9796d6f67b54c624, user_id=ac10e6a7a7a14cad855f64d8a16b1016, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:05.267059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4623206b-900c-4d2d-9638-22bed1e3617c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:05.277980 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1100/2209] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:09:05 2022] PUT /identity/v3/projects/24a782b8185d421d9796d6f67b54c624/users/ac10e6a7a7a14cad855f64d8a16b1016/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:05.288590 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b52f3-51b6-47af-ab2f-2e3b07c04f5a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:05.288931 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b52f3-51b6-47af-ab2f-2e3b07c04f5a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:05.289209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b52f3-51b6-47af-ab2f-2e3b07c04f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:05.314750 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e13b52f3-51b6-47af-ab2f-2e3b07c04f5a None None] MFA Rules not processed for user `ac10e6a7a7a14cad855f64d8a16b1016`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:05.317260 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e13b52f3-51b6-47af-ab2f-2e3b07c04f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:05.387277 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1110/2210] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:05 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:05.399331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623206b-900c-4d2d-9638-22bed1e3617c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:05.401237 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4623206b-900c-4d2d-9638-22bed1e3617c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:05.404486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:05.462187 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623206b-900c-4d2d-9638-22bed1e3617c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:05.465945 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6aa64-fc4f-42dd-a092-79c758f698d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:05.466632 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6aa64-fc4f-42dd-a092-79c758f698d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:05.467005 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6aa64-fc4f-42dd-a092-79c758f698d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:05.467376 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6aa64-fc4f-42dd-a092-79c758f698d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:05.479677 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d6aa64-fc4f-42dd-a092-79c758f698d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:05.479677 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d6aa64-fc4f-42dd-a092-79c758f698d6 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:05.488585 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1101/2211] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:09:05 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:09:05.675533 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b52f3-51b6-47af-ab2f-2e3b07c04f5a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:05.685296 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b52f3-51b6-47af-ab2f-2e3b07c04f5a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:05.688911 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cc8e9-6721-4bfd-95c4-a24b3bade3fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:05.689542 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cc8e9-6721-4bfd-95c4-a24b3bade3fa service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:05.689820 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cc8e9-6721-4bfd-95c4-a24b3bade3fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:05.690089 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cc8e9-6721-4bfd-95c4-a24b3bade3fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:05.746760 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526cc8e9-6721-4bfd-95c4-a24b3bade3fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:05.748600 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526cc8e9-6721-4bfd-95c4-a24b3bade3fa service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:05.758295 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1111/2212] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:09:05 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:09:05.770127 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:06.405093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:06.827367 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:07.021623 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:12.498184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6b3cc-5885-48c4-9247-21dd873fdd48 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:12.498707 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6b3cc-5885-48c4-9247-21dd873fdd48 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:12.498820 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6b3cc-5885-48c4-9247-21dd873fdd48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:12.517536 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1aa6b3cc-5885-48c4-9247-21dd873fdd48 None None] MFA Rules not processed for user `8a5b19c8c0d44e1f93bb1dea72884cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:12.519051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1aa6b3cc-5885-48c4-9247-21dd873fdd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:12.580456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1102/2213] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:13.228388 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cc8e9-6721-4bfd-95c4-a24b3bade3fa service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:13.230512 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-526cc8e9-6721-4bfd-95c4-a24b3bade3fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:13.341017 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cc8e9-6721-4bfd-95c4-a24b3bade3fa service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:13.346971 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f5a55-f91f-4b32-b1e1-6a95c1676499 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:13.349947 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f5a55-f91f-4b32-b1e1-6a95c1676499 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:13.349947 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f5a55-f91f-4b32-b1e1-6a95c1676499 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:13.349947 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f5a55-f91f-4b32-b1e1-6a95c1676499 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:13.351716 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-428f5a55-f91f-4b32-b1e1-6a95c1676499 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:13.453393 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f5a55-f91f-4b32-b1e1-6a95c1676499 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:13.454916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f5a55-f91f-4b32-b1e1-6a95c1676499 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:13.465140 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1112/2214] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:09:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:09:14.842948 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6b3cc-5885-48c4-9247-21dd873fdd48 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:14.844476 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1aa6b3cc-5885-48c4-9247-21dd873fdd48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:14.881278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:15.091146 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6b3cc-5885-48c4-9247-21dd873fdd48 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:15.099128 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38ac26-70e9-491c-b4b1-e0e8b3845cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:15.100341 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38ac26-70e9-491c-b4b1-e0e8b3845cb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:15.100667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38ac26-70e9-491c-b4b1-e0e8b3845cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:15.100948 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38ac26-70e9-491c-b4b1-e0e8b3845cb2 admin admin] PATH_INFO: `/v3/users/13cda543e1e5467983687551273c0e71` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:15.102337 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc38ac26-70e9-491c-b4b1-e0e8b3845cb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13cda543e1e5467983687551273c0e71)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:15.104505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc38ac26-70e9-491c-b4b1-e0e8b3845cb2 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:15.345616 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-bc38ac26-70e9-491c-b4b1-e0e8b3845cb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13cda543e1e5467983687551273c0e71'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:15.346524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-bc38ac26-70e9-491c-b4b1-e0e8b3845cb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13cda543e1e5467983687551273c0e71'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:15.390856 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-bc38ac26-70e9-491c-b4b1-e0e8b3845cb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13cda543e1e5467983687551273c0e71'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:15.400547 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-bc38ac26-70e9-491c-b4b1-e0e8b3845cb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13cda543e1e5467983687551273c0e71'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:15.541518 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-bc38ac26-70e9-491c-b4b1-e0e8b3845cb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13cda543e1e5467983687551273c0e71'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:15.547876 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1103/2215] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:09:14 2022] DELETE /identity/v3/users/13cda543e1e5467983687551273c0e71 => generated 0 bytes in 705 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:15.553266 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca852396-ac6a-4ede-96d0-26eba603dcbb None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:15.553755 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca852396-ac6a-4ede-96d0-26eba603dcbb None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:15.553847 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca852396-ac6a-4ede-96d0-26eba603dcbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:15.581236 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ca852396-ac6a-4ede-96d0-26eba603dcbb None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:15.583026 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ca852396-ac6a-4ede-96d0-26eba603dcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:15.632152 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:15.643567 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1113/2216] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:09:15.654384 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38ac26-70e9-491c-b4b1-e0e8b3845cb2 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:15.656124 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-bc38ac26-70e9-491c-b4b1-e0e8b3845cb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:15.713466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:15.765871 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38ac26-70e9-491c-b4b1-e0e8b3845cb2 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:15.770300 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e9d6c-dc25-432d-9f22-041c5c19d30b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:15.770957 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e9d6c-dc25-432d-9f22-041c5c19d30b service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:15.771240 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e9d6c-dc25-432d-9f22-041c5c19d30b service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:15.771514 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e9d6c-dc25-432d-9f22-041c5c19d30b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:15.783306 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e9d6c-dc25-432d-9f22-041c5c19d30b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:15.784960 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e9d6c-dc25-432d-9f22-041c5c19d30b service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:15.797390 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1104/2217] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:09:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:09:16.339122 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca852396-ac6a-4ede-96d0-26eba603dcbb None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:16.340854 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ca852396-ac6a-4ede-96d0-26eba603dcbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:16.355317 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca852396-ac6a-4ede-96d0-26eba603dcbb None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:16.361464 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d81cdf-bcb8-46a5-8d38-fda9d57527ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:16.362328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d81cdf-bcb8-46a5-8d38-fda9d57527ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:16.362651 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d81cdf-bcb8-46a5-8d38-fda9d57527ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:16.363003 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d81cdf-bcb8-46a5-8d38-fda9d57527ab admin admin] PATH_INFO: `/v3/projects/af7fdd651c2b4c8a8452e7b1d0889470` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:16.364787 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d81cdf-bcb8-46a5-8d38-fda9d57527ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=af7fdd651c2b4c8a8452e7b1d0889470)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:16.367217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d81cdf-bcb8-46a5-8d38-fda9d57527ab admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:16.506664 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d8d81cdf-bcb8-46a5-8d38-fda9d57527ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af7fdd651c2b4c8a8452e7b1d0889470', 'request_id': 'req-d8d81cdf-bcb8-46a5-8d38-fda9d57527ab'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:16.656113 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d8d81cdf-bcb8-46a5-8d38-fda9d57527ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af7fdd651c2b4c8a8452e7b1d0889470', 'request_id': 'req-d8d81cdf-bcb8-46a5-8d38-fda9d57527ab'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:16.663293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d8d81cdf-bcb8-46a5-8d38-fda9d57527ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af7fdd651c2b4c8a8452e7b1d0889470', 'request_id': 'req-d8d81cdf-bcb8-46a5-8d38-fda9d57527ab'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:16.674646 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d8d81cdf-bcb8-46a5-8d38-fda9d57527ab admin admin] The token cache is being invalidate because project af7fdd651c2b4c8a8452e7b1d0889470 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:09:16.674937 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d8d81cdf-bcb8-46a5-8d38-fda9d57527ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:16.675590 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d8d81cdf-bcb8-46a5-8d38-fda9d57527ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:16.677973 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1114/2218] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:09:16 2022] DELETE /identity/v3/projects/af7fdd651c2b4c8a8452e7b1d0889470 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:21.482028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa12fc7-a87b-46e6-8981-d9010de1b32d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:21.482028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa12fc7-a87b-46e6-8981-d9010de1b32d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:21.482028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa12fc7-a87b-46e6-8981-d9010de1b32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:21.502798 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6fa12fc7-a87b-46e6-8981-d9010de1b32d None None] MFA Rules not processed for user `a6a6e5f6e5f24ba89ca4895aebe158f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:21.504045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6fa12fc7-a87b-46e6-8981-d9010de1b32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:21.567821 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1105/2219] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:26.288334 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2316f26-4e38-4c2b-80ab-696c316394fa None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:26.289046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2316f26-4e38-4c2b-80ab-696c316394fa None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:26.289433 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2316f26-4e38-4c2b-80ab-696c316394fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:26.331044 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e2316f26-4e38-4c2b-80ab-696c316394fa None None] MFA Rules not processed for user `1d37f369749143ce813c693adc5f2ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:26.332469 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e2316f26-4e38-4c2b-80ab-696c316394fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:26.412718 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1115/2220] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jun 23 20:09:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:26.458125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa12fc7-a87b-46e6-8981-d9010de1b32d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:26.459663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6fa12fc7-a87b-46e6-8981-d9010de1b32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:26.506780 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2316f26-4e38-4c2b-80ab-696c316394fa None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:26.508491 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e2316f26-4e38-4c2b-80ab-696c316394fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:26.579708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa12fc7-a87b-46e6-8981-d9010de1b32d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:26.585372 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce209a4d-bc20-4147-9d8f-3138185ae75d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96db03079816461bb322633f64b9fb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:26.586083 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce209a4d-bc20-4147-9d8f-3138185ae75d service swift] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:26.586363 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce209a4d-bc20-4147-9d8f-3138185ae75d service swift] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:26.586637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce209a4d-bc20-4147-9d8f-3138185ae75d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:26.597388 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce209a4d-bc20-4147-9d8f-3138185ae75d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:26.600759 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2316f26-4e38-4c2b-80ab-696c316394fa None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:26.600959 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce209a4d-bc20-4147-9d8f-3138185ae75d service swift] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:26.607873 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd7a68-5175-4b77-bd4b-da574df873ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:26.608991 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd7a68-5175-4b77-bd4b-da574df873ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:26.609279 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd7a68-5175-4b77-bd4b-da574df873ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:26.609571 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdd7a68-5175-4b77-bd4b-da574df873ac admin admin] PATH_INFO: `/v3/users/5435ccbb0c954a96b7e393c509e9ea70` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:26.610931 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd7a68-5175-4b77-bd4b-da574df873ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=5435ccbb0c954a96b7e393c509e9ea70)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:26.611094 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1106/2221] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jun 23 20:09:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 20:09:26.613347 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdd7a68-5175-4b77-bd4b-da574df873ac admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:26.650017 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:26.830312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6cdd7a68-5175-4b77-bd4b-da574df873ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5435ccbb0c954a96b7e393c509e9ea70'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:26.833882 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6cdd7a68-5175-4b77-bd4b-da574df873ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5435ccbb0c954a96b7e393c509e9ea70'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:26.931222 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6cdd7a68-5175-4b77-bd4b-da574df873ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5435ccbb0c954a96b7e393c509e9ea70'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:26.931901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6cdd7a68-5175-4b77-bd4b-da574df873ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5435ccbb0c954a96b7e393c509e9ea70'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:26.953070 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6cdd7a68-5175-4b77-bd4b-da574df873ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5435ccbb0c954a96b7e393c509e9ea70'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:26.981715 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1116/2222] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:09:26 2022] DELETE /identity/v3/users/5435ccbb0c954a96b7e393c509e9ea70 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:09:26.986630 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7440bb4-2b5a-4671-85a4-33f4842d278c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:26.986919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7440bb4-2b5a-4671-85a4-33f4842d278c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:26.987194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7440bb4-2b5a-4671-85a4-33f4842d278c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:27.021717 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a7440bb4-2b5a-4671-85a4-33f4842d278c None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:27.025662 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a7440bb4-2b5a-4671-85a4-33f4842d278c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:27.136597 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:27.179600 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1107/2223] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:27.208341 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd7a68-5175-4b77-bd4b-da574df873ac admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:27.211072 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6cdd7a68-5175-4b77-bd4b-da574df873ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:27.288385 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdd7a68-5175-4b77-bd4b-da574df873ac admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:27.294061 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5d643-3151-4d27-b68d-266ecc687a48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:27.294716 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5d643-3151-4d27-b68d-266ecc687a48 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:27.294996 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5d643-3151-4d27-b68d-266ecc687a48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:27.295351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5d643-3151-4d27-b68d-266ecc687a48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:27.309670 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5d643-3151-4d27-b68d-266ecc687a48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:27.309670 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5d643-3151-4d27-b68d-266ecc687a48 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:27.323572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1117/2224] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:09:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:09:28.759452 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7440bb4-2b5a-4671-85a4-33f4842d278c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:28.761491 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a7440bb4-2b5a-4671-85a4-33f4842d278c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:28.779373 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7440bb4-2b5a-4671-85a4-33f4842d278c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:28.804807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da0847-720a-49b4-8324-84d06fe55eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:28.813422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da0847-720a-49b4-8324-84d06fe55eec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:28.813422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da0847-720a-49b4-8324-84d06fe55eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:28.813929 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da0847-720a-49b4-8324-84d06fe55eec admin admin] PATH_INFO: `/v3/projects/e608a3985b6e46bcafaebe462fa253be` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:28.817143 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da0847-720a-49b4-8324-84d06fe55eec admin admin] RBAC: Authorizing `identity:delete_project(project_id=e608a3985b6e46bcafaebe462fa253be)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:28.820627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da0847-720a-49b4-8324-84d06fe55eec admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:28.973863 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d4da0847-720a-49b4-8324-84d06fe55eec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e608a3985b6e46bcafaebe462fa253be', 'request_id': 'req-d4da0847-720a-49b4-8324-84d06fe55eec'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:28.987694 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d4da0847-720a-49b4-8324-84d06fe55eec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e608a3985b6e46bcafaebe462fa253be', 'request_id': 'req-d4da0847-720a-49b4-8324-84d06fe55eec'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:29.001780 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d4da0847-720a-49b4-8324-84d06fe55eec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e608a3985b6e46bcafaebe462fa253be', 'request_id': 'req-d4da0847-720a-49b4-8324-84d06fe55eec'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:29.139814 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d4da0847-720a-49b4-8324-84d06fe55eec admin admin] The token cache is being invalidate because project e608a3985b6e46bcafaebe462fa253be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:09:29.140343 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d4da0847-720a-49b4-8324-84d06fe55eec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:29.141049 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d4da0847-720a-49b4-8324-84d06fe55eec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:29.142963 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1108/2225] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:09:28 2022] DELETE /identity/v3/projects/e608a3985b6e46bcafaebe462fa253be => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:29.148011 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5d643-3151-4d27-b68d-266ecc687a48 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:29.162480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5d643-3151-4d27-b68d-266ecc687a48 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:29.266675 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73863905-297e-4136-9743-ca8240bf46fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:29.267850 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73863905-297e-4136-9743-ca8240bf46fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:29.267850 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73863905-297e-4136-9743-ca8240bf46fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:29.268128 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73863905-297e-4136-9743-ca8240bf46fe admin admin] PATH_INFO: `/v3/users/ac10e6a7a7a14cad855f64d8a16b1016` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:29.277303 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73863905-297e-4136-9743-ca8240bf46fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac10e6a7a7a14cad855f64d8a16b1016)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:29.284729 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73863905-297e-4136-9743-ca8240bf46fe admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:29.498839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:29.513349 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-73863905-297e-4136-9743-ca8240bf46fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac10e6a7a7a14cad855f64d8a16b1016'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:29.514281 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-73863905-297e-4136-9743-ca8240bf46fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac10e6a7a7a14cad855f64d8a16b1016'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:29.589974 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-73863905-297e-4136-9743-ca8240bf46fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac10e6a7a7a14cad855f64d8a16b1016'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:29.593065 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-73863905-297e-4136-9743-ca8240bf46fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac10e6a7a7a14cad855f64d8a16b1016'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:29.630883 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-73863905-297e-4136-9743-ca8240bf46fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac10e6a7a7a14cad855f64d8a16b1016'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:29.655252 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1118/2226] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:09:29 2022] DELETE /identity/v3/users/ac10e6a7a7a14cad855f64d8a16b1016 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:30.825219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da0847-720a-49b4-8324-84d06fe55eec admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:30.826828 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d4da0847-720a-49b4-8324-84d06fe55eec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:30.842169 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da0847-720a-49b4-8324-84d06fe55eec admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:30.915560 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a3b5fe-59a4-4ace-aea2-eeaca2143f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:30.916611 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3b5fe-59a4-4ace-aea2-eeaca2143f12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:30.916885 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3b5fe-59a4-4ace-aea2-eeaca2143f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:30.917164 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3b5fe-59a4-4ace-aea2-eeaca2143f12 admin admin] PATH_INFO: `/v3/projects/24a782b8185d421d9796d6f67b54c624` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:30.919126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a3b5fe-59a4-4ace-aea2-eeaca2143f12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24a782b8185d421d9796d6f67b54c624)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:30.921190 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a3b5fe-59a4-4ace-aea2-eeaca2143f12 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:31.043149 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c4a3b5fe-59a4-4ace-aea2-eeaca2143f12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24a782b8185d421d9796d6f67b54c624', 'request_id': 'req-c4a3b5fe-59a4-4ace-aea2-eeaca2143f12'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:31.049781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c4a3b5fe-59a4-4ace-aea2-eeaca2143f12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24a782b8185d421d9796d6f67b54c624', 'request_id': 'req-c4a3b5fe-59a4-4ace-aea2-eeaca2143f12'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:31.061222 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c4a3b5fe-59a4-4ace-aea2-eeaca2143f12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24a782b8185d421d9796d6f67b54c624', 'request_id': 'req-c4a3b5fe-59a4-4ace-aea2-eeaca2143f12'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:31.151315 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c4a3b5fe-59a4-4ace-aea2-eeaca2143f12 admin admin] The token cache is being invalidate because project 24a782b8185d421d9796d6f67b54c624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:09:31.154036 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c4a3b5fe-59a4-4ace-aea2-eeaca2143f12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:31.154036 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c4a3b5fe-59a4-4ace-aea2-eeaca2143f12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:31.155110 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1109/2227] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:09:30 2022] DELETE /identity/v3/projects/24a782b8185d421d9796d6f67b54c624 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:31.160926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ed6c0-ad22-4a5a-bd23-f6082192e887 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:31.161888 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ed6c0-ad22-4a5a-bd23-f6082192e887 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:31.162406 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ed6c0-ad22-4a5a-bd23-f6082192e887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:31.261732 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a70ed6c0-ad22-4a5a-bd23-f6082192e887 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:31.263281 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a70ed6c0-ad22-4a5a-bd23-f6082192e887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:31.350016 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1119/2228] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:31.358137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcaeb1-e7af-41a9-9b4b-cf9e5c7288ab None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:31.358662 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcaeb1-e7af-41a9-9b4b-cf9e5c7288ab None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:31.359158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcaeb1-e7af-41a9-9b4b-cf9e5c7288ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:31.371330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:31.395778 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6bbcaeb1-e7af-41a9-9b4b-cf9e5c7288ab None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:31.397039 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6bbcaeb1-e7af-41a9-9b4b-cf9e5c7288ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:31.409461 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1110/2229] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:31.413512 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ed6c0-ad22-4a5a-bd23-f6082192e887 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:31.423348 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ed6c0-ad22-4a5a-bd23-f6082192e887 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:31.429689 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c2a8a-3856-41db-80ac-e2376820d1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:31.430605 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c2a8a-3856-41db-80ac-e2376820d1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:31.430999 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c2a8a-3856-41db-80ac-e2376820d1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:31.433542 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c2a8a-3856-41db-80ac-e2376820d1f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:31.434420 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c2a8a-3856-41db-80ac-e2376820d1f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:09:31.434799 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c2a8a-3856-41db-80ac-e2376820d1f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:31.436691 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c2a8a-3856-41db-80ac-e2376820d1f1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:31.450915 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1120/2230] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:09:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:09:31.455402 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcaeb1-e7af-41a9-9b4b-cf9e5c7288ab None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:31.465387 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcaeb1-e7af-41a9-9b4b-cf9e5c7288ab None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:31.470675 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2deb7-6f88-423d-ab9f-79f997a42152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:31.471568 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2deb7-6f88-423d-ab9f-79f997a42152 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:31.471893 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2deb7-6f88-423d-ab9f-79f997a42152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:31.472389 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2deb7-6f88-423d-ab9f-79f997a42152 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:31.474294 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b2deb7-6f88-423d-ab9f-79f997a42152 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:31.475224 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b2deb7-6f88-423d-ab9f-79f997a42152 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:31.609447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c2a8a-3856-41db-80ac-e2376820d1f1 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:31.611532 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-aa4c2a8a-3856-41db-80ac-e2376820d1f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:31.625081 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c2a8a-3856-41db-80ac-e2376820d1f1 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:31.630898 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c72a5-8597-43ee-95ef-9aeba8989945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:31.631871 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c72a5-8597-43ee-95ef-9aeba8989945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:31.632306 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c72a5-8597-43ee-95ef-9aeba8989945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:31.632699 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c72a5-8597-43ee-95ef-9aeba8989945 admin admin] PATH_INFO: `/v3/users/68f69257b4c94da4b7370e6a5cbfe544` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:31.636260 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c72a5-8597-43ee-95ef-9aeba8989945 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68f69257b4c94da4b7370e6a5cbfe544)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:31.638397 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c72a5-8597-43ee-95ef-9aeba8989945 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:31.712360 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1111/2231] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:09:31 2022] POST /identity/v3/projects => generated 385 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:09:31.714805 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2deb7-6f88-423d-ab9f-79f997a42152 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:31.800074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2deb7-6f88-423d-ab9f-79f997a42152 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:31.807045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca650c-4ea2-44cf-8e1b-27b780100129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:31.807738 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca650c-4ea2-44cf-8e1b-27b780100129 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:31.808022 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca650c-4ea2-44cf-8e1b-27b780100129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:31.808329 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca650c-4ea2-44cf-8e1b-27b780100129 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:31.809035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca650c-4ea2-44cf-8e1b-27b780100129 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:31.811064 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca650c-4ea2-44cf-8e1b-27b780100129 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:31.867168 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fe1c72a5-8597-43ee-95ef-9aeba8989945 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68f69257b4c94da4b7370e6a5cbfe544'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:31.867859 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fe1c72a5-8597-43ee-95ef-9aeba8989945 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68f69257b4c94da4b7370e6a5cbfe544'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:31.922394 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1112/2232] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:09:31 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:09:31.923568 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca650c-4ea2-44cf-8e1b-27b780100129 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:31.934167 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca650c-4ea2-44cf-8e1b-27b780100129 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:31.945100 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f6164-f153-4b2b-845b-8a86fbbbe5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:31.945100 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f6164-f153-4b2b-845b-8a86fbbbe5df admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:31.945396 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f6164-f153-4b2b-845b-8a86fbbbe5df admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:31.945633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f6164-f153-4b2b-845b-8a86fbbbe5df admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:31.949008 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f6164-f153-4b2b-845b-8a86fbbbe5df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:31.953044 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f6164-f153-4b2b-845b-8a86fbbbe5df admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:31.971220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fe1c72a5-8597-43ee-95ef-9aeba8989945 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68f69257b4c94da4b7370e6a5cbfe544'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:31.971796 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fe1c72a5-8597-43ee-95ef-9aeba8989945 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68f69257b4c94da4b7370e6a5cbfe544'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:31.975845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1113/2233] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:09:31 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:09:31.980579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f6164-f153-4b2b-845b-8a86fbbbe5df admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:31.982710 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-cd8f6164-f153-4b2b-845b-8a86fbbbe5df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:31.998744 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f6164-f153-4b2b-845b-8a86fbbbe5df admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:32.006117 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cddd85-53b6-459e-aa7c-cf0c61f3bd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:32.008068 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cddd85-53b6-459e-aa7c-cf0c61f3bd55 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:32.016046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fe1c72a5-8597-43ee-95ef-9aeba8989945 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68f69257b4c94da4b7370e6a5cbfe544'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:32.016856 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cddd85-53b6-459e-aa7c-cf0c61f3bd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:32.017465 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cddd85-53b6-459e-aa7c-cf0c61f3bd55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:32.018400 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cddd85-53b6-459e-aa7c-cf0c61f3bd55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:32.019171 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:32.021378 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cddd85-53b6-459e-aa7c-cf0c61f3bd55 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:32.037222 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1114/2234] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:09:31 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:09:32.041178 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cddd85-53b6-459e-aa7c-cf0c61f3bd55 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:32.043509 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1121/2235] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:09:31 2022] DELETE /identity/v3/users/68f69257b4c94da4b7370e6a5cbfe544 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:09:32.053905 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc4bb1-ad5b-4520-8e76-a654a65ae92b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:32.054192 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc4bb1-ad5b-4520-8e76-a654a65ae92b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:32.054476 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc4bb1-ad5b-4520-8e76-a654a65ae92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:32.062016 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cddd85-53b6-459e-aa7c-cf0c61f3bd55 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:32.100021 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-efbc4bb1-ad5b-4520-8e76-a654a65ae92b None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:32.101323 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-efbc4bb1-ad5b-4520-8e76-a654a65ae92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:32.150627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9eeba0-ad08-445c-a4c3-5d7c4026b50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:32.151844 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9eeba0-ad08-445c-a4c3-5d7c4026b50b admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:32.152129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9eeba0-ad08-445c-a4c3-5d7c4026b50b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:32.152574 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9eeba0-ad08-445c-a4c3-5d7c4026b50b admin admin] PATH_INFO: `/v3/projects/21065890600e44369b941feabb691551/users/7d00f4c4f5384542a710fd25d30849b1/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:32.184809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9eeba0-ad08-445c-a4c3-5d7c4026b50b admin admin] RBAC: Authorizing `identity:create_grant(project_id=21065890600e44369b941feabb691551, user_id=7d00f4c4f5384542a710fd25d30849b1, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:32.187833 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9eeba0-ad08-445c-a4c3-5d7c4026b50b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:32.200339 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1122/2236] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:32.208473 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc4bb1-ad5b-4520-8e76-a654a65ae92b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:32.209905 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-efbc4bb1-ad5b-4520-8e76-a654a65ae92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:32.282874 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1115/2237] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:09:32 2022] PUT /identity/v3/projects/21065890600e44369b941feabb691551/users/7d00f4c4f5384542a710fd25d30849b1/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:32.293998 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b9bf7-49e5-4579-9a3d-c354d9448e45 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:32.294549 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b9bf7-49e5-4579-9a3d-c354d9448e45 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:32.295349 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b9bf7-49e5-4579-9a3d-c354d9448e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:32.302321 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc4bb1-ad5b-4520-8e76-a654a65ae92b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:32.365635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-875b9bf7-49e5-4579-9a3d-c354d9448e45 None None] MFA Rules not processed for user `7d00f4c4f5384542a710fd25d30849b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:32.367695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-875b9bf7-49e5-4579-9a3d-c354d9448e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:32.377463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b92819-a00d-47a9-8139-3fd00a7fb2f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:32.378131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b92819-a00d-47a9-8139-3fd00a7fb2f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:32.378558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b92819-a00d-47a9-8139-3fd00a7fb2f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:32.378845 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b92819-a00d-47a9-8139-3fd00a7fb2f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:32.389786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:32.475832 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1116/2238] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:09:32.479026 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b9bf7-49e5-4579-9a3d-c354d9448e45 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:32.516922 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b92819-a00d-47a9-8139-3fd00a7fb2f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:32.519191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b92819-a00d-47a9-8139-3fd00a7fb2f2 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:32.528592 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:32.536372 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1123/2239] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:09:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:09:32.601974 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b9bf7-49e5-4579-9a3d-c354d9448e45 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:32.610238 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325310db-077f-416c-8e27-7d13664403e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:32.611241 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325310db-077f-416c-8e27-7d13664403e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:32.611555 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325310db-077f-416c-8e27-7d13664403e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:32.611834 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325310db-077f-416c-8e27-7d13664403e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:32.612574 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325310db-077f-416c-8e27-7d13664403e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:32.614527 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325310db-077f-416c-8e27-7d13664403e5 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:32.770734 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1117/2240] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:09:32 2022] POST /identity/v3/projects => generated 387 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:09:32.774589 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b92819-a00d-47a9-8139-3fd00a7fb2f2 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:32.785093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b92819-a00d-47a9-8139-3fd00a7fb2f2 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:32.883591 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd530b9-9fe9-46fe-aba0-ca136b309bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:32.884286 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd530b9-9fe9-46fe-aba0-ca136b309bb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:32.884614 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd530b9-9fe9-46fe-aba0-ca136b309bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:32.884895 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd530b9-9fe9-46fe-aba0-ca136b309bb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:32.885607 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd530b9-9fe9-46fe-aba0-ca136b309bb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:32.887606 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd530b9-9fe9-46fe-aba0-ca136b309bb0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:33.007781 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1124/2241] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:09:32 2022] POST /identity/v3/users => generated 300 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:09:33.009520 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325310db-077f-416c-8e27-7d13664403e5 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:33.026980 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325310db-077f-416c-8e27-7d13664403e5 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:33.034695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624b048-c170-407d-80e1-fe445801eefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:33.036471 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624b048-c170-407d-80e1-fe445801eefa admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:33.037270 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624b048-c170-407d-80e1-fe445801eefa admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:33.038346 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e624b048-c170-407d-80e1-fe445801eefa admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:33.039912 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624b048-c170-407d-80e1-fe445801eefa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:33.041802 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e624b048-c170-407d-80e1-fe445801eefa admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:33.066906 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1118/2242] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:09:33.068157 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd530b9-9fe9-46fe-aba0-ca136b309bb0 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:33.084205 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd530b9-9fe9-46fe-aba0-ca136b309bb0 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:33.090925 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe818a8-7f7e-4e8f-9801-90826e20439d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:33.092330 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe818a8-7f7e-4e8f-9801-90826e20439d admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:33.092621 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe818a8-7f7e-4e8f-9801-90826e20439d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:33.093007 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe818a8-7f7e-4e8f-9801-90826e20439d admin admin] PATH_INFO: `/v3/projects/ec8cfa4f1f184137b1eb8cf63d80471b/users/febfee10f640495da237b838088c7b60/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:33.124818 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe818a8-7f7e-4e8f-9801-90826e20439d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec8cfa4f1f184137b1eb8cf63d80471b, user_id=febfee10f640495da237b838088c7b60, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:33.127852 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe818a8-7f7e-4e8f-9801-90826e20439d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:33.201145 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1125/2243] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:09:33 2022] PUT /identity/v3/projects/ec8cfa4f1f184137b1eb8cf63d80471b/users/febfee10f640495da237b838088c7b60/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:33.205177 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624b048-c170-407d-80e1-fe445801eefa admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:33.224404 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e624b048-c170-407d-80e1-fe445801eefa admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:33.250627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe818a8-7f7e-4e8f-9801-90826e20439d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:33.252506 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8fe818a8-7f7e-4e8f-9801-90826e20439d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:33.363132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30802122-1a63-4ec4-a0e6-09f9f7843359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:33.363959 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30802122-1a63-4ec4-a0e6-09f9f7843359 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:33.371159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30802122-1a63-4ec4-a0e6-09f9f7843359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:33.371159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30802122-1a63-4ec4-a0e6-09f9f7843359 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:33.371159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30802122-1a63-4ec4-a0e6-09f9f7843359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:33.371159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30802122-1a63-4ec4-a0e6-09f9f7843359 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:33.384654 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1119/2244] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:09:33.391084 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe818a8-7f7e-4e8f-9801-90826e20439d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:33.391616 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30802122-1a63-4ec4-a0e6-09f9f7843359 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:33.408722 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70027cd1-7fd4-42a9-b861-b719695c7502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:33.410401 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70027cd1-7fd4-42a9-b861-b719695c7502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:33.410588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30802122-1a63-4ec4-a0e6-09f9f7843359 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:33.411212 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70027cd1-7fd4-42a9-b861-b719695c7502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:33.411817 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70027cd1-7fd4-42a9-b861-b719695c7502 admin admin] PATH_INFO: `/v3/projects/6563af64e83446be985451f9b66bb112` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:33.414478 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70027cd1-7fd4-42a9-b861-b719695c7502 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6563af64e83446be985451f9b66bb112)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:33.415810 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4ac5e-c8cf-4596-9a8f-7cef771e3094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:33.418626 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70027cd1-7fd4-42a9-b861-b719695c7502 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:33.419010 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4ac5e-c8cf-4596-9a8f-7cef771e3094 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:33.419366 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4ac5e-c8cf-4596-9a8f-7cef771e3094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:33.419713 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4ac5e-c8cf-4596-9a8f-7cef771e3094 admin admin] PATH_INFO: `/v3/projects/ec8cfa4f1f184137b1eb8cf63d80471b/users/febfee10f640495da237b838088c7b60/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:33.422133 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4ac5e-c8cf-4596-9a8f-7cef771e3094 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec8cfa4f1f184137b1eb8cf63d80471b, user_id=febfee10f640495da237b838088c7b60, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:33.425198 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4ac5e-c8cf-4596-9a8f-7cef771e3094 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:33.437604 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1120/2245] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:09:33 2022] PUT /identity/v3/projects/ec8cfa4f1f184137b1eb8cf63d80471b/users/febfee10f640495da237b838088c7b60/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:33.452199 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4a355-a00c-4337-8d7e-89ba915edc58 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:33.455548 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4a355-a00c-4337-8d7e-89ba915edc58 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:33.455548 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4a355-a00c-4337-8d7e-89ba915edc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:33.479581 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:33.488548 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-15d4a355-a00c-4337-8d7e-89ba915edc58 None None] MFA Rules not processed for user `febfee10f640495da237b838088c7b60`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:33.490106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-15d4a355-a00c-4337-8d7e-89ba915edc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:33.559089 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-70027cd1-7fd4-42a9-b861-b719695c7502 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6563af64e83446be985451f9b66bb112', 'request_id': 'req-70027cd1-7fd4-42a9-b861-b719695c7502'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:33.586086 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1121/2246] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:33.601394 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4a355-a00c-4337-8d7e-89ba915edc58 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:33.602947 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-15d4a355-a00c-4337-8d7e-89ba915edc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:33.679055 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-70027cd1-7fd4-42a9-b861-b719695c7502 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6563af64e83446be985451f9b66bb112', 'request_id': 'req-70027cd1-7fd4-42a9-b861-b719695c7502'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:33.680749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4a355-a00c-4337-8d7e-89ba915edc58 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:33.687141 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-70027cd1-7fd4-42a9-b861-b719695c7502 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6563af64e83446be985451f9b66bb112', 'request_id': 'req-70027cd1-7fd4-42a9-b861-b719695c7502'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:33.692282 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:33.718430 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-70027cd1-7fd4-42a9-b861-b719695c7502 admin admin] The token cache is being invalidate because project 6563af64e83446be985451f9b66bb112 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:09:33.718780 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-70027cd1-7fd4-42a9-b861-b719695c7502 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:33.719547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-70027cd1-7fd4-42a9-b861-b719695c7502 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:33.721769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1126/2247] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:09:33 2022] DELETE /identity/v3/projects/6563af64e83446be985451f9b66bb112 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:33.730470 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370241d8-9495-4a0c-b33d-2e7b89d9fa93 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:33.730755 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370241d8-9495-4a0c-b33d-2e7b89d9fa93 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:33.731947 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370241d8-9495-4a0c-b33d-2e7b89d9fa93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:33.772364 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417fe67d-fa9d-4bbc-a9db-1b02fea2c3cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:33.773060 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fe67d-fa9d-4bbc-a9db-1b02fea2c3cd service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:33.773342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fe67d-fa9d-4bbc-a9db-1b02fea2c3cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:33.773619 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417fe67d-fa9d-4bbc-a9db-1b02fea2c3cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:33.774138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-370241d8-9495-4a0c-b33d-2e7b89d9fa93 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:33.775559 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-417fe67d-fa9d-4bbc-a9db-1b02fea2c3cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:33.775993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-370241d8-9495-4a0c-b33d-2e7b89d9fa93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:33.869519 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1127/2248] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:33.878733 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb777b-3b41-4c8c-a7cf-846a3a06acf6 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:33.879982 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb777b-3b41-4c8c-a7cf-846a3a06acf6 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:33.879982 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb777b-3b41-4c8c-a7cf-846a3a06acf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:33.904087 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-acfb777b-3b41-4c8c-a7cf-846a3a06acf6 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:33.907086 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-acfb777b-3b41-4c8c-a7cf-846a3a06acf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:33.920328 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417fe67d-fa9d-4bbc-a9db-1b02fea2c3cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:33.921866 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417fe67d-fa9d-4bbc-a9db-1b02fea2c3cd service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:33.924083 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1128/2249] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:33.928888 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb777b-3b41-4c8c-a7cf-846a3a06acf6 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:33.950065 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb777b-3b41-4c8c-a7cf-846a3a06acf6 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:33.962033 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea6fde8-25bd-4601-b5e0-9eb07714f575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:33.962779 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6fde8-25bd-4601-b5e0-9eb07714f575 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:33.963129 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6fde8-25bd-4601-b5e0-9eb07714f575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:33.963477 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6fde8-25bd-4601-b5e0-9eb07714f575 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:33.964312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea6fde8-25bd-4601-b5e0-9eb07714f575 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:09:33.964717 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea6fde8-25bd-4601-b5e0-9eb07714f575 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:33.966537 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea6fde8-25bd-4601-b5e0-9eb07714f575 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:33.968163 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1122/2250] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:09:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:09:33.985031 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1129/2251] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:09:33.986770 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417fe67d-fa9d-4bbc-a9db-1b02fea2c3cd service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:33.997168 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:34.006379 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417fe67d-fa9d-4bbc-a9db-1b02fea2c3cd service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:34.018982 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea6fde8-25bd-4601-b5e0-9eb07714f575 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:34.020915 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6ea6fde8-25bd-4601-b5e0-9eb07714f575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:34.021379 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34388199-b1ec-461b-b819-1570ba92ea8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:34.022129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34388199-b1ec-461b-b819-1570ba92ea8a admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:34.022527 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34388199-b1ec-461b-b819-1570ba92ea8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:34.022915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34388199-b1ec-461b-b819-1570ba92ea8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:34.023717 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34388199-b1ec-461b-b819-1570ba92ea8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:34.025934 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34388199-b1ec-461b-b819-1570ba92ea8a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:34.044089 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea6fde8-25bd-4601-b5e0-9eb07714f575 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:34.069150 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02cf258-0ef0-4163-b505-08ec02bdfa32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:34.071764 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02cf258-0ef0-4163-b505-08ec02bdfa32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:34.072417 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02cf258-0ef0-4163-b505-08ec02bdfa32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:34.073617 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02cf258-0ef0-4163-b505-08ec02bdfa32 admin admin] PATH_INFO: `/v3/users/7d00f4c4f5384542a710fd25d30849b1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:34.076740 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02cf258-0ef0-4163-b505-08ec02bdfa32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d00f4c4f5384542a710fd25d30849b1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:34.090002 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02cf258-0ef0-4163-b505-08ec02bdfa32 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:34.144062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1123/2252] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:09:33 2022] POST /identity/v3/projects => generated 375 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:09:34.149188 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34388199-b1ec-461b-b819-1570ba92ea8a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:34.248291 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34388199-b1ec-461b-b819-1570ba92ea8a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:34.248674 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15a773-54fb-49e0-b1c0-2aaa15eccd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:34.249353 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15a773-54fb-49e0-b1c0-2aaa15eccd97 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:34.249687 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15a773-54fb-49e0-b1c0-2aaa15eccd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:34.250021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b15a773-54fb-49e0-b1c0-2aaa15eccd97 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:34.250881 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b15a773-54fb-49e0-b1c0-2aaa15eccd97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:34.252898 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b15a773-54fb-49e0-b1c0-2aaa15eccd97 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:34.503617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1124/2253] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:09:34 2022] POST /identity/v3/users => generated 288 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:09:34.507597 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15a773-54fb-49e0-b1c0-2aaa15eccd97 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:34.517842 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b15a773-54fb-49e0-b1c0-2aaa15eccd97 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:34.523026 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33fc72e-d7b7-47d4-a9a2-aa5ee9412a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:34.523680 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33fc72e-d7b7-47d4-a9a2-aa5ee9412a1a admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:34.523987 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33fc72e-d7b7-47d4-a9a2-aa5ee9412a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:34.524450 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33fc72e-d7b7-47d4-a9a2-aa5ee9412a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:34.525054 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33fc72e-d7b7-47d4-a9a2-aa5ee9412a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:34.526976 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33fc72e-d7b7-47d4-a9a2-aa5ee9412a1a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:34.538747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1125/2254] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:09:34.543499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33fc72e-d7b7-47d4-a9a2-aa5ee9412a1a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:34.553595 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33fc72e-d7b7-47d4-a9a2-aa5ee9412a1a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:34.558164 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ece38-54b7-41eb-bad8-5e5105f9a288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:34.558824 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ece38-54b7-41eb-bad8-5e5105f9a288 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:34.559089 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ece38-54b7-41eb-bad8-5e5105f9a288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:34.559370 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ece38-54b7-41eb-bad8-5e5105f9a288 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:34.560136 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ece38-54b7-41eb-bad8-5e5105f9a288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:34.564148 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00ece38-54b7-41eb-bad8-5e5105f9a288 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:34.578202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1126/2255] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:09:34.579237 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ece38-54b7-41eb-bad8-5e5105f9a288 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:34.598721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ece38-54b7-41eb-bad8-5e5105f9a288 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:34.603348 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a6c52c-17ef-47e3-8943-43a4b379fa9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:34.604219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6c52c-17ef-47e3-8943-43a4b379fa9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:34.604530 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6c52c-17ef-47e3-8943-43a4b379fa9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:34.604802 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a6c52c-17ef-47e3-8943-43a4b379fa9a admin admin] PATH_INFO: `/v3/projects/fee2271744814fe9b2726faa89cfe1bc/users/3457dad643d34950834cc41462aa412b/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:34.619389 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6c52c-17ef-47e3-8943-43a4b379fa9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fee2271744814fe9b2726faa89cfe1bc, user_id=3457dad643d34950834cc41462aa412b, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:34.622100 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a6c52c-17ef-47e3-8943-43a4b379fa9a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:34.778244 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a02cf258-0ef0-4163-b505-08ec02bdfa32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d00f4c4f5384542a710fd25d30849b1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:34.778956 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a02cf258-0ef0-4163-b505-08ec02bdfa32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d00f4c4f5384542a710fd25d30849b1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:34.936101 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1127/2256] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:09:34 2022] PUT /identity/v3/projects/fee2271744814fe9b2726faa89cfe1bc/users/3457dad643d34950834cc41462aa412b/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:09:34.942259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a02cf258-0ef0-4163-b505-08ec02bdfa32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d00f4c4f5384542a710fd25d30849b1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:34.945399 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a02cf258-0ef0-4163-b505-08ec02bdfa32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d00f4c4f5384542a710fd25d30849b1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:34.946542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0049585-857c-40ac-bcc3-bf54b6a7be23 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:34.946840 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0049585-857c-40ac-bcc3-bf54b6a7be23 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:34.947120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0049585-857c-40ac-bcc3-bf54b6a7be23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:34.960702 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a02cf258-0ef0-4163-b505-08ec02bdfa32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d00f4c4f5384542a710fd25d30849b1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:34.981905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a0049585-857c-40ac-bcc3-bf54b6a7be23 None None] MFA Rules not processed for user `3457dad643d34950834cc41462aa412b`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:34.981905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a0049585-857c-40ac-bcc3-bf54b6a7be23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:34.982783 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1130/2257] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:09:34 2022] DELETE /identity/v3/users/7d00f4c4f5384542a710fd25d30849b1 => generated 0 bytes in 961 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:34.984552 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15cd8bc-257a-4aae-9c9a-5c5f2f9dd027 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:34.984845 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15cd8bc-257a-4aae-9c9a-5c5f2f9dd027 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:34.985124 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15cd8bc-257a-4aae-9c9a-5c5f2f9dd027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:35.025411 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e15cd8bc-257a-4aae-9c9a-5c5f2f9dd027 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:35.026674 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e15cd8bc-257a-4aae-9c9a-5c5f2f9dd027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:35.076140 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1128/2258] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:35.093026 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0049585-857c-40ac-bcc3-bf54b6a7be23 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:35.093026 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a0049585-857c-40ac-bcc3-bf54b6a7be23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:35.094772 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:35.111550 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:35.153453 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1131/2259] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:35.167875 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15cd8bc-257a-4aae-9c9a-5c5f2f9dd027 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:35.171953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e15cd8bc-257a-4aae-9c9a-5c5f2f9dd027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:35.173904 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0049585-857c-40ac-bcc3-bf54b6a7be23 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:35.178895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e44eaa-1b28-4452-afe8-05e9fa3770cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:35.179784 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e44eaa-1b28-4452-afe8-05e9fa3770cb service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:35.180260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e44eaa-1b28-4452-afe8-05e9fa3770cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:35.180670 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e44eaa-1b28-4452-afe8-05e9fa3770cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:35.191147 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e44eaa-1b28-4452-afe8-05e9fa3770cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:35.193950 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e44eaa-1b28-4452-afe8-05e9fa3770cb service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:35.205489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1129/2260] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:09:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:09:35.290923 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15cd8bc-257a-4aae-9c9a-5c5f2f9dd027 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:35.300563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be35a1-0343-463d-b6fc-213395a9c224 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:35.300563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be35a1-0343-463d-b6fc-213395a9c224 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:35.300563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be35a1-0343-463d-b6fc-213395a9c224 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:35.300563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be35a1-0343-463d-b6fc-213395a9c224 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:35.308080 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be35a1-0343-463d-b6fc-213395a9c224 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:35.310790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be35a1-0343-463d-b6fc-213395a9c224 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:35.330746 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1132/2261] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:09:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:09:36.111584 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e44eaa-1b28-4452-afe8-05e9fa3770cb service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:36.113155 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f7e44eaa-1b28-4452-afe8-05e9fa3770cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:36.166352 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e44eaa-1b28-4452-afe8-05e9fa3770cb service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:36.184073 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27970e9c-c5c1-4e5e-bb01-662e3a891a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:36.185238 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27970e9c-c5c1-4e5e-bb01-662e3a891a49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:36.185522 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27970e9c-c5c1-4e5e-bb01-662e3a891a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:36.185823 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27970e9c-c5c1-4e5e-bb01-662e3a891a49 admin admin] PATH_INFO: `/v3/projects/21065890600e44369b941feabb691551` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:36.187618 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27970e9c-c5c1-4e5e-bb01-662e3a891a49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21065890600e44369b941feabb691551)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:36.189748 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27970e9c-c5c1-4e5e-bb01-662e3a891a49 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:36.314745 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-27970e9c-c5c1-4e5e-bb01-662e3a891a49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21065890600e44369b941feabb691551', 'request_id': 'req-27970e9c-c5c1-4e5e-bb01-662e3a891a49'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:36.322761 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-27970e9c-c5c1-4e5e-bb01-662e3a891a49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21065890600e44369b941feabb691551', 'request_id': 'req-27970e9c-c5c1-4e5e-bb01-662e3a891a49'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:36.331682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-27970e9c-c5c1-4e5e-bb01-662e3a891a49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21065890600e44369b941feabb691551', 'request_id': 'req-27970e9c-c5c1-4e5e-bb01-662e3a891a49'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:36.422522 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-27970e9c-c5c1-4e5e-bb01-662e3a891a49 admin admin] The token cache is being invalidate because project 21065890600e44369b941feabb691551 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:09:36.423128 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-27970e9c-c5c1-4e5e-bb01-662e3a891a49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:36.423801 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-27970e9c-c5c1-4e5e-bb01-662e3a891a49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:36.425406 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1130/2262] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:09:36 2022] DELETE /identity/v3/projects/21065890600e44369b941feabb691551 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:36.429441 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be35a1-0343-463d-b6fc-213395a9c224 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:36.451789 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be35a1-0343-463d-b6fc-213395a9c224 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:36.554284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f9c0d-881b-4df9-bc42-603848ccda89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:36.557245 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f9c0d-881b-4df9-bc42-603848ccda89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:36.557245 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f9c0d-881b-4df9-bc42-603848ccda89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:36.557245 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f9c0d-881b-4df9-bc42-603848ccda89 admin admin] PATH_INFO: `/v3/users/febfee10f640495da237b838088c7b60` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:36.565154 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29f9c0d-881b-4df9-bc42-603848ccda89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=febfee10f640495da237b838088c7b60)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:36.565549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29f9c0d-881b-4df9-bc42-603848ccda89 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:36.795998 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d29f9c0d-881b-4df9-bc42-603848ccda89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'febfee10f640495da237b838088c7b60'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:36.797252 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d29f9c0d-881b-4df9-bc42-603848ccda89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'febfee10f640495da237b838088c7b60'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:36.907920 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d29f9c0d-881b-4df9-bc42-603848ccda89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'febfee10f640495da237b838088c7b60'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:36.908778 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d29f9c0d-881b-4df9-bc42-603848ccda89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'febfee10f640495da237b838088c7b60'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:36.931795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d29f9c0d-881b-4df9-bc42-603848ccda89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'febfee10f640495da237b838088c7b60'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:36.953900 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1133/2263] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:09:36 2022] DELETE /identity/v3/users/febfee10f640495da237b838088c7b60 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:37.095814 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc9c5e-0822-451c-b3f0-f260833a330e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:37.096265 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc9c5e-0822-451c-b3f0-f260833a330e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:37.096674 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dc9c5e-0822-451c-b3f0-f260833a330e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:37.121559 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-85dc9c5e-0822-451c-b3f0-f260833a330e None None] MFA Rules not processed for user `3457dad643d34950834cc41462aa412b`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:37.124495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-85dc9c5e-0822-451c-b3f0-f260833a330e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:37.194476 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1131/2264] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:37.709945 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f9c0d-881b-4df9-bc42-603848ccda89 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:37.711990 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d29f9c0d-881b-4df9-bc42-603848ccda89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:37.729565 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:37.821574 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f9c0d-881b-4df9-bc42-603848ccda89 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:37.827340 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae569f57-09f1-4880-bd7b-05ce2c798bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:37.828242 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae569f57-09f1-4880-bd7b-05ce2c798bf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:37.828539 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae569f57-09f1-4880-bd7b-05ce2c798bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:37.828944 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae569f57-09f1-4880-bd7b-05ce2c798bf2 admin admin] PATH_INFO: `/v3/projects/ec8cfa4f1f184137b1eb8cf63d80471b` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:37.830298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae569f57-09f1-4880-bd7b-05ce2c798bf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec8cfa4f1f184137b1eb8cf63d80471b)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:37.833189 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae569f57-09f1-4880-bd7b-05ce2c798bf2 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:38.011946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ae569f57-09f1-4880-bd7b-05ce2c798bf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec8cfa4f1f184137b1eb8cf63d80471b', 'request_id': 'req-ae569f57-09f1-4880-bd7b-05ce2c798bf2'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:38.149269 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ae569f57-09f1-4880-bd7b-05ce2c798bf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec8cfa4f1f184137b1eb8cf63d80471b', 'request_id': 'req-ae569f57-09f1-4880-bd7b-05ce2c798bf2'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:38.161551 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ae569f57-09f1-4880-bd7b-05ce2c798bf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec8cfa4f1f184137b1eb8cf63d80471b', 'request_id': 'req-ae569f57-09f1-4880-bd7b-05ce2c798bf2'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:38.182266 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ae569f57-09f1-4880-bd7b-05ce2c798bf2 admin admin] The token cache is being invalidate because project ec8cfa4f1f184137b1eb8cf63d80471b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:09:38.184175 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ae569f57-09f1-4880-bd7b-05ce2c798bf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:38.184974 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ae569f57-09f1-4880-bd7b-05ce2c798bf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:38.190614 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1134/2265] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:09:37 2022] DELETE /identity/v3/projects/ec8cfa4f1f184137b1eb8cf63d80471b => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:38.192352 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335cc21-f0d5-4992-b8b6-f4d8ffecb38f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:38.192835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335cc21-f0d5-4992-b8b6-f4d8ffecb38f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:38.192920 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9335cc21-f0d5-4992-b8b6-f4d8ffecb38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:38.252245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9335cc21-f0d5-4992-b8b6-f4d8ffecb38f None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:38.253516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9335cc21-f0d5-4992-b8b6-f4d8ffecb38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:38.347499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:38.387981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1132/2266] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:38.395971 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df942f4d-d9f2-45ed-9b92-2b0080d8a80c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:38.397187 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df942f4d-d9f2-45ed-9b92-2b0080d8a80c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:38.397697 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df942f4d-d9f2-45ed-9b92-2b0080d8a80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:38.421124 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-df942f4d-d9f2-45ed-9b92-2b0080d8a80c None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:38.423273 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-df942f4d-d9f2-45ed-9b92-2b0080d8a80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:38.447237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1135/2267] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:38.448196 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335cc21-f0d5-4992-b8b6-f4d8ffecb38f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:38.465430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9335cc21-f0d5-4992-b8b6-f4d8ffecb38f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:38.470195 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbd724-c84e-4980-8473-85544885f0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:38.470984 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbd724-c84e-4980-8473-85544885f0e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:38.471335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbd724-c84e-4980-8473-85544885f0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:38.471728 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbd724-c84e-4980-8473-85544885f0e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:38.472599 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbd724-c84e-4980-8473-85544885f0e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:09:38.474187 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbd724-c84e-4980-8473-85544885f0e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:38.476981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbd724-c84e-4980-8473-85544885f0e2 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:38.496236 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1133/2268] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:09:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 20:09:38.502719 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df942f4d-d9f2-45ed-9b92-2b0080d8a80c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:38.509075 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df942f4d-d9f2-45ed-9b92-2b0080d8a80c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:38.514081 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1b7aa-a600-4b34-8026-105394085aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:38.514770 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1b7aa-a600-4b34-8026-105394085aab admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:38.515047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1b7aa-a600-4b34-8026-105394085aab admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:38.515315 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1b7aa-a600-4b34-8026-105394085aab admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:38.516023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e1b7aa-a600-4b34-8026-105394085aab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:38.518097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e1b7aa-a600-4b34-8026-105394085aab admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:39.344256 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1136/2269] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:09:38 2022] POST /identity/v3/projects => generated 383 bytes in 847 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:09:39.349138 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbd724-c84e-4980-8473-85544885f0e2 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:39.364184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbd724-c84e-4980-8473-85544885f0e2 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:39.466502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0a508-cd1a-4c3c-92c2-605e169632e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:39.467206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0a508-cd1a-4c3c-92c2-605e169632e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:39.467494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0a508-cd1a-4c3c-92c2-605e169632e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:39.468107 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0a508-cd1a-4c3c-92c2-605e169632e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:39.469012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b0a508-cd1a-4c3c-92c2-605e169632e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:39.471349 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b0a508-cd1a-4c3c-92c2-605e169632e3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:39.572184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1134/2270] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:09:39 2022] POST /identity/v3/users => generated 292 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:09:39.576214 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1b7aa-a600-4b34-8026-105394085aab admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:39.592792 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1b7aa-a600-4b34-8026-105394085aab admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:39.598613 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679373a8-b9b5-4912-bd27-83ba361e7dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:39.599458 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679373a8-b9b5-4912-bd27-83ba361e7dfb admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:39.599740 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679373a8-b9b5-4912-bd27-83ba361e7dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:39.600013 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679373a8-b9b5-4912-bd27-83ba361e7dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:39.613070 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679373a8-b9b5-4912-bd27-83ba361e7dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:39.614917 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679373a8-b9b5-4912-bd27-83ba361e7dfb admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:39.632047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1137/2271] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:09:39.635920 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0a508-cd1a-4c3c-92c2-605e169632e3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:39.649609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0a508-cd1a-4c3c-92c2-605e169632e3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:39.660121 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d102a9c-8da1-41d5-bf3e-e6589cbd7e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:39.667816 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d102a9c-8da1-41d5-bf3e-e6589cbd7e39 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:39.667816 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d102a9c-8da1-41d5-bf3e-e6589cbd7e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:39.667816 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d102a9c-8da1-41d5-bf3e-e6589cbd7e39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:39.667816 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d102a9c-8da1-41d5-bf3e-e6589cbd7e39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:39.667816 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d102a9c-8da1-41d5-bf3e-e6589cbd7e39 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:39.680019 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1135/2272] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:09:39.684871 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679373a8-b9b5-4912-bd27-83ba361e7dfb admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:39.701365 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679373a8-b9b5-4912-bd27-83ba361e7dfb admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:39.712991 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c374e9-85ec-458e-b364-1885caa77eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:39.715921 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c374e9-85ec-458e-b364-1885caa77eaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:39.716260 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c374e9-85ec-458e-b364-1885caa77eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:39.717976 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c374e9-85ec-458e-b364-1885caa77eaa admin admin] PATH_INFO: `/v3/projects/1febc535abd64f2fb47115cfc69498c3/users/c57fd3be26394552b50a4dff6362ad62/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:39.784706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c374e9-85ec-458e-b364-1885caa77eaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=1febc535abd64f2fb47115cfc69498c3, user_id=c57fd3be26394552b50a4dff6362ad62, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:39.805775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c374e9-85ec-458e-b364-1885caa77eaa admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:39.885109 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1138/2273] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:09:39 2022] PUT /identity/v3/projects/1febc535abd64f2fb47115cfc69498c3/users/c57fd3be26394552b50a4dff6362ad62/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:39.895763 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f8c07-f43b-4e9d-8242-1cdd65c1c35e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:39.896048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f8c07-f43b-4e9d-8242-1cdd65c1c35e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:39.896341 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f8c07-f43b-4e9d-8242-1cdd65c1c35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:39.950080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e18f8c07-f43b-4e9d-8242-1cdd65c1c35e None None] MFA Rules not processed for user `c57fd3be26394552b50a4dff6362ad62`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:39.951936 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e18f8c07-f43b-4e9d-8242-1cdd65c1c35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:40.072404 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1136/2274] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:40.077409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c374e9-85ec-458e-b364-1885caa77eaa admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:40.127533 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:40.156208 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c374e9-85ec-458e-b364-1885caa77eaa admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:40.161992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4c79e-8172-4981-8d13-e574827f74cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:40.162811 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4c79e-8172-4981-8d13-e574827f74cc admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:40.163099 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4c79e-8172-4981-8d13-e574827f74cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:40.163431 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4c79e-8172-4981-8d13-e574827f74cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:40.164196 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4c79e-8172-4981-8d13-e574827f74cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:40.166251 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4c79e-8172-4981-8d13-e574827f74cc admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:40.243568 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1139/2275] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:09:40 2022] POST /identity/v3/projects => generated 381 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:09:40.247682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f8c07-f43b-4e9d-8242-1cdd65c1c35e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:40.262497 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f8c07-f43b-4e9d-8242-1cdd65c1c35e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:40.303450 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:40.336586 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd95a10-94e9-436e-afb1-a30b3e13871a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:40.337275 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd95a10-94e9-436e-afb1-a30b3e13871a admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:40.337555 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd95a10-94e9-436e-afb1-a30b3e13871a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:40.337829 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd95a10-94e9-436e-afb1-a30b3e13871a admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:40.338543 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd95a10-94e9-436e-afb1-a30b3e13871a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:40.341764 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd95a10-94e9-436e-afb1-a30b3e13871a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:40.559823 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1137/2276] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:09:40 2022] POST /identity/v3/users => generated 297 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:09:40.564067 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4c79e-8172-4981-8d13-e574827f74cc admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:40.575387 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4c79e-8172-4981-8d13-e574827f74cc admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:40.582489 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f07c6-28b6-4224-bf29-b45e8078adc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:40.585616 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f07c6-28b6-4224-bf29-b45e8078adc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:40.585616 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f07c6-28b6-4224-bf29-b45e8078adc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:40.585616 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f07c6-28b6-4224-bf29-b45e8078adc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:40.585616 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f07c6-28b6-4224-bf29-b45e8078adc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:40.586060 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f07c6-28b6-4224-bf29-b45e8078adc0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:40.598586 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1140/2277] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:09:40.602868 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd95a10-94e9-436e-afb1-a30b3e13871a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:40.618702 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd95a10-94e9-436e-afb1-a30b3e13871a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:40.620746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ebcf1a-2f94-4717-ad51-d85b1d1ef42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:40.622294 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebcf1a-2f94-4717-ad51-d85b1d1ef42e admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:40.622663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebcf1a-2f94-4717-ad51-d85b1d1ef42e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:40.623025 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebcf1a-2f94-4717-ad51-d85b1d1ef42e admin admin] PATH_INFO: `/v3/projects/bbb9c1ccf3a8486791ca7eb94d266641/users/5966510a75f54e31aa5490176df857c9/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:40.642270 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ebcf1a-2f94-4717-ad51-d85b1d1ef42e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbb9c1ccf3a8486791ca7eb94d266641, user_id=5966510a75f54e31aa5490176df857c9, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:40.645209 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ebcf1a-2f94-4717-ad51-d85b1d1ef42e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:40.764520 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f07c6-28b6-4224-bf29-b45e8078adc0 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:40.764697 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1138/2278] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:09:40 2022] PUT /identity/v3/projects/bbb9c1ccf3a8486791ca7eb94d266641/users/5966510a75f54e31aa5490176df857c9/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:09:40.778107 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f07c6-28b6-4224-bf29-b45e8078adc0 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:40.868342 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71f70c-8e15-4757-ba66-ee9e4766a309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:40.868342 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71f70c-8e15-4757-ba66-ee9e4766a309 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:40.868690 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71f70c-8e15-4757-ba66-ee9e4766a309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:40.868960 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71f70c-8e15-4757-ba66-ee9e4766a309 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:40.869762 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe71f70c-8e15-4757-ba66-ee9e4766a309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:40.871583 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe71f70c-8e15-4757-ba66-ee9e4766a309 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:40.884181 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1141/2279] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:09:40.888391 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ebcf1a-2f94-4717-ad51-d85b1d1ef42e admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:40.906322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ebcf1a-2f94-4717-ad51-d85b1d1ef42e admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:40.913124 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c360202-871a-4bae-a65f-7748df91d2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:40.914019 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c360202-871a-4bae-a65f-7748df91d2fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:40.914307 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c360202-871a-4bae-a65f-7748df91d2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:40.914585 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c360202-871a-4bae-a65f-7748df91d2fd admin admin] PATH_INFO: `/v3/projects/bbb9c1ccf3a8486791ca7eb94d266641/users/5966510a75f54e31aa5490176df857c9/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:40.916670 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c360202-871a-4bae-a65f-7748df91d2fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbb9c1ccf3a8486791ca7eb94d266641, user_id=5966510a75f54e31aa5490176df857c9, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:40.919393 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c360202-871a-4bae-a65f-7748df91d2fd admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:40.944570 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1139/2280] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:09:40 2022] PUT /identity/v3/projects/bbb9c1ccf3a8486791ca7eb94d266641/users/5966510a75f54e31aa5490176df857c9/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:40.952480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958ea92-fe93-44b0-b98c-a220ed344cbe None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:40.952738 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958ea92-fe93-44b0-b98c-a220ed344cbe None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:40.953016 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d958ea92-fe93-44b0-b98c-a220ed344cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:40.999868 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d958ea92-fe93-44b0-b98c-a220ed344cbe None None] MFA Rules not processed for user `5966510a75f54e31aa5490176df857c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:41.001373 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d958ea92-fe93-44b0-b98c-a220ed344cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:41.099521 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1142/2281] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:40 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:41.111244 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c360202-871a-4bae-a65f-7748df91d2fd admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:41.113319 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0c360202-871a-4bae-a65f-7748df91d2fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:41.195733 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c360202-871a-4bae-a65f-7748df91d2fd admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:41.199934 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdc8d89-6edb-4a50-bc7d-e903710d1d08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:41.200613 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc8d89-6edb-4a50-bc7d-e903710d1d08 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:41.200896 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc8d89-6edb-4a50-bc7d-e903710d1d08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:41.201234 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc8d89-6edb-4a50-bc7d-e903710d1d08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:41.213454 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdc8d89-6edb-4a50-bc7d-e903710d1d08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:41.214930 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdc8d89-6edb-4a50-bc7d-e903710d1d08 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:41.225966 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1140/2282] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:09:41 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:09:41.339916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958ea92-fe93-44b0-b98c-a220ed344cbe None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:41.349611 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d958ea92-fe93-44b0-b98c-a220ed344cbe None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:41.353177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab397d-07d0-4d1f-812e-5c0c3c118ad1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:41.353818 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab397d-07d0-4d1f-812e-5c0c3c118ad1 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:41.354102 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab397d-07d0-4d1f-812e-5c0c3c118ad1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:41.354374 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab397d-07d0-4d1f-812e-5c0c3c118ad1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:41.385628 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:41.414878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab397d-07d0-4d1f-812e-5c0c3c118ad1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:41.417036 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab397d-07d0-4d1f-812e-5c0c3c118ad1 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:41.427280 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1143/2283] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:09:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:09:44.480125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdc8d89-6edb-4a50-bc7d-e903710d1d08 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:44.541748 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdc8d89-6edb-4a50-bc7d-e903710d1d08 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:44.546248 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7cd9dc-2aa0-46db-b6ae-a86c42f558c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:44.547102 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7cd9dc-2aa0-46db-b6ae-a86c42f558c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:44.547384 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7cd9dc-2aa0-46db-b6ae-a86c42f558c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:44.547654 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7cd9dc-2aa0-46db-b6ae-a86c42f558c9 admin admin] PATH_INFO: `/v3/users/c57fd3be26394552b50a4dff6362ad62` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:44.548862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7cd9dc-2aa0-46db-b6ae-a86c42f558c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c57fd3be26394552b50a4dff6362ad62)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:44.550768 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7cd9dc-2aa0-46db-b6ae-a86c42f558c9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:44.689859 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:44.698057 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4a7cd9dc-2aa0-46db-b6ae-a86c42f558c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c57fd3be26394552b50a4dff6362ad62'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:44.698999 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4a7cd9dc-2aa0-46db-b6ae-a86c42f558c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c57fd3be26394552b50a4dff6362ad62'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:44.711064 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4a7cd9dc-2aa0-46db-b6ae-a86c42f558c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c57fd3be26394552b50a4dff6362ad62'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:44.717758 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4a7cd9dc-2aa0-46db-b6ae-a86c42f558c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c57fd3be26394552b50a4dff6362ad62'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:44.770398 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4a7cd9dc-2aa0-46db-b6ae-a86c42f558c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c57fd3be26394552b50a4dff6362ad62'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:44.776482 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1141/2284] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:09:44 2022] DELETE /identity/v3/users/c57fd3be26394552b50a4dff6362ad62 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:44.781105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e4030-2566-42ce-8126-c008332ddd6e None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:44.781604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e4030-2566-42ce-8126-c008332ddd6e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:44.781709 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e4030-2566-42ce-8126-c008332ddd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:44.801673 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-507e4030-2566-42ce-8126-c008332ddd6e None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:44.801673 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-507e4030-2566-42ce-8126-c008332ddd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:44.857325 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1144/2285] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:44.866910 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7cd9dc-2aa0-46db-b6ae-a86c42f558c9 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:44.868279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4a7cd9dc-2aa0-46db-b6ae-a86c42f558c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:44.924712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7cd9dc-2aa0-46db-b6ae-a86c42f558c9 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:44.928396 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e1e20-18df-44f6-98ad-ae8f5f6118d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:44.929040 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e1e20-18df-44f6-98ad-ae8f5f6118d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:44.929318 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e1e20-18df-44f6-98ad-ae8f5f6118d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:44.929591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0e1e20-18df-44f6-98ad-ae8f5f6118d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:44.938887 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e1e20-18df-44f6-98ad-ae8f5f6118d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:44.940344 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0e1e20-18df-44f6-98ad-ae8f5f6118d5 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:44.950658 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1142/2286] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:09:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:09:45.609715 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e4030-2566-42ce-8126-c008332ddd6e None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:45.611080 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-507e4030-2566-42ce-8126-c008332ddd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:45.623549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e4030-2566-42ce-8126-c008332ddd6e None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:45.628139 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6586b4-9a1b-4153-9182-e63759fad0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:45.629024 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6586b4-9a1b-4153-9182-e63759fad0fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:45.629385 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6586b4-9a1b-4153-9182-e63759fad0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:45.629646 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6586b4-9a1b-4153-9182-e63759fad0fa admin admin] PATH_INFO: `/v3/projects/1febc535abd64f2fb47115cfc69498c3` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:45.630831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6586b4-9a1b-4153-9182-e63759fad0fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=1febc535abd64f2fb47115cfc69498c3)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:45.632783 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6586b4-9a1b-4153-9182-e63759fad0fa admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:45.730915 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2c6586b4-9a1b-4153-9182-e63759fad0fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1febc535abd64f2fb47115cfc69498c3', 'request_id': 'req-2c6586b4-9a1b-4153-9182-e63759fad0fa'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:45.781619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2c6586b4-9a1b-4153-9182-e63759fad0fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1febc535abd64f2fb47115cfc69498c3', 'request_id': 'req-2c6586b4-9a1b-4153-9182-e63759fad0fa'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:45.787973 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2c6586b4-9a1b-4153-9182-e63759fad0fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1febc535abd64f2fb47115cfc69498c3', 'request_id': 'req-2c6586b4-9a1b-4153-9182-e63759fad0fa'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:45.798916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2c6586b4-9a1b-4153-9182-e63759fad0fa admin admin] The token cache is being invalidate because project 1febc535abd64f2fb47115cfc69498c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:09:45.799198 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2c6586b4-9a1b-4153-9182-e63759fad0fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:45.799815 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2c6586b4-9a1b-4153-9182-e63759fad0fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:45.801925 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1145/2287] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:09:45 2022] DELETE /identity/v3/projects/1febc535abd64f2fb47115cfc69498c3 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:09:45.805980 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e1e20-18df-44f6-98ad-ae8f5f6118d5 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:45.815627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0e1e20-18df-44f6-98ad-ae8f5f6118d5 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:45.876829 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ca9f3-2924-40d4-b59d-f4d70877a891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:45.877709 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ca9f3-2924-40d4-b59d-f4d70877a891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:45.877986 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ca9f3-2924-40d4-b59d-f4d70877a891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:45.878307 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ca9f3-2924-40d4-b59d-f4d70877a891 admin admin] PATH_INFO: `/v3/users/5966510a75f54e31aa5490176df857c9` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:45.879819 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ca9f3-2924-40d4-b59d-f4d70877a891 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5966510a75f54e31aa5490176df857c9)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:45.881909 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ca9f3-2924-40d4-b59d-f4d70877a891 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:46.045881 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a14ca9f3-2924-40d4-b59d-f4d70877a891 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5966510a75f54e31aa5490176df857c9'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:46.046548 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a14ca9f3-2924-40d4-b59d-f4d70877a891 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5966510a75f54e31aa5490176df857c9'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:46.055068 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:46.059542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a14ca9f3-2924-40d4-b59d-f4d70877a891 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5966510a75f54e31aa5490176df857c9'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:46.066367 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a14ca9f3-2924-40d4-b59d-f4d70877a891 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5966510a75f54e31aa5490176df857c9'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:46.159924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a14ca9f3-2924-40d4-b59d-f4d70877a891 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5966510a75f54e31aa5490176df857c9'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:46.166947 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1143/2288] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:09:45 2022] DELETE /identity/v3/users/5966510a75f54e31aa5490176df857c9 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:46.732412 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6586b4-9a1b-4153-9182-e63759fad0fa admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:46.733869 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2c6586b4-9a1b-4153-9182-e63759fad0fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:46.747270 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6586b4-9a1b-4153-9182-e63759fad0fa admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:46.798325 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:46.807639 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e0206-dec5-4ed9-952f-ccdddf1de7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:46.808551 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e0206-dec5-4ed9-952f-ccdddf1de7a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:46.808836 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e0206-dec5-4ed9-952f-ccdddf1de7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:46.809127 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e0206-dec5-4ed9-952f-ccdddf1de7a8 admin admin] PATH_INFO: `/v3/projects/bbb9c1ccf3a8486791ca7eb94d266641` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:46.810340 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e0206-dec5-4ed9-952f-ccdddf1de7a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbb9c1ccf3a8486791ca7eb94d266641)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:46.812392 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2e0206-dec5-4ed9-952f-ccdddf1de7a8 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:46.924803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dc2e0206-dec5-4ed9-952f-ccdddf1de7a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbb9c1ccf3a8486791ca7eb94d266641', 'request_id': 'req-dc2e0206-dec5-4ed9-952f-ccdddf1de7a8'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:47.039379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dc2e0206-dec5-4ed9-952f-ccdddf1de7a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbb9c1ccf3a8486791ca7eb94d266641', 'request_id': 'req-dc2e0206-dec5-4ed9-952f-ccdddf1de7a8'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:47.045642 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dc2e0206-dec5-4ed9-952f-ccdddf1de7a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbb9c1ccf3a8486791ca7eb94d266641', 'request_id': 'req-dc2e0206-dec5-4ed9-952f-ccdddf1de7a8'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:47.056742 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dc2e0206-dec5-4ed9-952f-ccdddf1de7a8 admin admin] The token cache is being invalidate because project bbb9c1ccf3a8486791ca7eb94d266641 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:09:47.058196 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dc2e0206-dec5-4ed9-952f-ccdddf1de7a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:47.058196 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dc2e0206-dec5-4ed9-952f-ccdddf1de7a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:47.059143 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1146/2289] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:09:46 2022] DELETE /identity/v3/projects/bbb9c1ccf3a8486791ca7eb94d266641 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:47.064074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14cd5d-1f55-4336-92e8-89359dad627b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:47.064602 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14cd5d-1f55-4336-92e8-89359dad627b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:47.064836 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14cd5d-1f55-4336-92e8-89359dad627b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:47.085385 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1d14cd5d-1f55-4336-92e8-89359dad627b None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:47.086817 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1d14cd5d-1f55-4336-92e8-89359dad627b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:47.149936 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1144/2290] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:47.157919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b450f2d-242f-4e42-8d22-1e9a12852344 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:47.158316 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b450f2d-242f-4e42-8d22-1e9a12852344 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:47.158677 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b450f2d-242f-4e42-8d22-1e9a12852344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:47.181996 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-1b450f2d-242f-4e42-8d22-1e9a12852344 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:47.183249 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1b450f2d-242f-4e42-8d22-1e9a12852344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:47.191297 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1147/2291] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:47.196222 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14cd5d-1f55-4336-92e8-89359dad627b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:47.206772 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14cd5d-1f55-4336-92e8-89359dad627b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:47.212220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7774f-b980-4920-9372-8ecf63f08ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:47.213534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7774f-b980-4920-9372-8ecf63f08ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:47.213928 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7774f-b980-4920-9372-8ecf63f08ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:47.214206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7774f-b980-4920-9372-8ecf63f08ce8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:47.215265 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7774f-b980-4920-9372-8ecf63f08ce8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:09:47.215573 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7774f-b980-4920-9372-8ecf63f08ce8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:47.217234 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7774f-b980-4920-9372-8ecf63f08ce8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:47.237364 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1145/2292] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:09:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:09:47.241850 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b450f2d-242f-4e42-8d22-1e9a12852344 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:47.260259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b450f2d-242f-4e42-8d22-1e9a12852344 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:47.265990 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af2d40-8e51-4089-ac98-5205deda0a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:47.285340 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af2d40-8e51-4089-ac98-5205deda0a03 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:47.285340 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af2d40-8e51-4089-ac98-5205deda0a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:47.285340 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af2d40-8e51-4089-ac98-5205deda0a03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:47.285340 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af2d40-8e51-4089-ac98-5205deda0a03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:47.285340 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af2d40-8e51-4089-ac98-5205deda0a03 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:47.433097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1148/2293] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:09:47 2022] POST /identity/v3/projects => generated 379 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:09:47.443268 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7774f-b980-4920-9372-8ecf63f08ce8 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:47.447875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7774f-b980-4920-9372-8ecf63f08ce8 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:47.516393 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd56807-2646-4cee-8266-6669bf0ebe25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:47.517071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd56807-2646-4cee-8266-6669bf0ebe25 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:47.517353 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd56807-2646-4cee-8266-6669bf0ebe25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:47.517624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd56807-2646-4cee-8266-6669bf0ebe25 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:47.518306 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd56807-2646-4cee-8266-6669bf0ebe25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:47.520206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd56807-2646-4cee-8266-6669bf0ebe25 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:47.648938 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1146/2294] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:09:47 2022] POST /identity/v3/users => generated 290 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:09:47.653065 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af2d40-8e51-4089-ac98-5205deda0a03 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:47.660954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af2d40-8e51-4089-ac98-5205deda0a03 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:47.665567 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02a621-b39d-4919-ac4c-c69ac46462cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:47.666304 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02a621-b39d-4919-ac4c-c69ac46462cf admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:47.666582 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02a621-b39d-4919-ac4c-c69ac46462cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:47.666853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa02a621-b39d-4919-ac4c-c69ac46462cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:47.667581 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa02a621-b39d-4919-ac4c-c69ac46462cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:47.669796 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa02a621-b39d-4919-ac4c-c69ac46462cf admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:47.684359 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1149/2295] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:09:47.685562 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd56807-2646-4cee-8266-6669bf0ebe25 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:47.697325 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd56807-2646-4cee-8266-6669bf0ebe25 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:47.702293 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39c260-ec69-4ba2-b095-89274432f94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:47.702955 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39c260-ec69-4ba2-b095-89274432f94c admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:47.703232 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39c260-ec69-4ba2-b095-89274432f94c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:47.703504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39c260-ec69-4ba2-b095-89274432f94c admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:47.704227 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb39c260-ec69-4ba2-b095-89274432f94c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:47.705855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb39c260-ec69-4ba2-b095-89274432f94c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:47.716511 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1147/2296] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:09:47.724096 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02a621-b39d-4919-ac4c-c69ac46462cf admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:47.734976 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa02a621-b39d-4919-ac4c-c69ac46462cf admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:47.741768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28471c45-0525-4762-a471-cacfaae7a864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:47.742662 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28471c45-0525-4762-a471-cacfaae7a864 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:47.743031 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28471c45-0525-4762-a471-cacfaae7a864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:47.743303 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28471c45-0525-4762-a471-cacfaae7a864 admin admin] PATH_INFO: `/v3/projects/6f02996c3b16439a80a20a126066b024/users/1e69b6aceee244b890c0f623dbd8d390/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:47.772809 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28471c45-0525-4762-a471-cacfaae7a864 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f02996c3b16439a80a20a126066b024, user_id=1e69b6aceee244b890c0f623dbd8d390, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:47.776513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28471c45-0525-4762-a471-cacfaae7a864 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:47.889238 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1150/2297] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:09:47 2022] PUT /identity/v3/projects/6f02996c3b16439a80a20a126066b024/users/1e69b6aceee244b890c0f623dbd8d390/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:09:47.891613 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcc2e6-5bef-4971-ba67-d8d3272d22cc None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:47.891903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcc2e6-5bef-4971-ba67-d8d3272d22cc None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:47.892177 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcc2e6-5bef-4971-ba67-d8d3272d22cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:47.920361 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-bbdcc2e6-5bef-4971-ba67-d8d3272d22cc None None] MFA Rules not processed for user `1e69b6aceee244b890c0f623dbd8d390`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:47.922319 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-bbdcc2e6-5bef-4971-ba67-d8d3272d22cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:47.954580 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:47.993017 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1148/2298] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:47.997508 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28471c45-0525-4762-a471-cacfaae7a864 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:48.063847 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28471c45-0525-4762-a471-cacfaae7a864 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:48.070837 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d41f6-b25a-42ed-b0ef-9b75001c1eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:48.071674 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d41f6-b25a-42ed-b0ef-9b75001c1eae admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:48.072070 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d41f6-b25a-42ed-b0ef-9b75001c1eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:48.072444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949d41f6-b25a-42ed-b0ef-9b75001c1eae admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:48.073248 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d41f6-b25a-42ed-b0ef-9b75001c1eae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:48.076845 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949d41f6-b25a-42ed-b0ef-9b75001c1eae admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:48.212657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1151/2299] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:09:47 2022] POST /identity/v3/projects => generated 379 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:09:48.224831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcc2e6-5bef-4971-ba67-d8d3272d22cc None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:48.238675 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcc2e6-5bef-4971-ba67-d8d3272d22cc None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:48.299249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5aadb-f276-409b-9473-7973862664f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:48.299926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5aadb-f276-409b-9473-7973862664f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:48.300208 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5aadb-f276-409b-9473-7973862664f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:48.300529 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5aadb-f276-409b-9473-7973862664f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:48.301461 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5aadb-f276-409b-9473-7973862664f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:48.303546 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5aadb-f276-409b-9473-7973862664f7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:48.417346 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1149/2300] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:09:48 2022] POST /identity/v3/users => generated 290 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:09:48.421248 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d41f6-b25a-42ed-b0ef-9b75001c1eae admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:48.427955 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:48.431571 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949d41f6-b25a-42ed-b0ef-9b75001c1eae admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:48.436314 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29afdeb-3e33-47bd-bb41-1a9dac62d8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:48.436957 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29afdeb-3e33-47bd-bb41-1a9dac62d8af admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:48.437250 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29afdeb-3e33-47bd-bb41-1a9dac62d8af admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:48.437558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29afdeb-3e33-47bd-bb41-1a9dac62d8af admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:48.438282 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29afdeb-3e33-47bd-bb41-1a9dac62d8af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:48.440045 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29afdeb-3e33-47bd-bb41-1a9dac62d8af admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:48.452154 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1152/2301] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:09:48.459637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5aadb-f276-409b-9473-7973862664f7 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:48.469858 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5aadb-f276-409b-9473-7973862664f7 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:48.475418 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b06a93-dc0d-47ae-8fd5-da144be036a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:48.476077 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b06a93-dc0d-47ae-8fd5-da144be036a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:48.476383 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b06a93-dc0d-47ae-8fd5-da144be036a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:48.476656 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b06a93-dc0d-47ae-8fd5-da144be036a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:48.477380 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b06a93-dc0d-47ae-8fd5-da144be036a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:48.478966 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b06a93-dc0d-47ae-8fd5-da144be036a6 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:48.489090 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1150/2302] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:09:48.493238 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29afdeb-3e33-47bd-bb41-1a9dac62d8af admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:48.503011 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29afdeb-3e33-47bd-bb41-1a9dac62d8af admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:48.507455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb78dd-9d00-47a1-a5e9-01988e78219d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:48.508341 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb78dd-9d00-47a1-a5e9-01988e78219d admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:48.508620 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb78dd-9d00-47a1-a5e9-01988e78219d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:48.508889 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb78dd-9d00-47a1-a5e9-01988e78219d admin admin] PATH_INFO: `/v3/projects/b2d423182a3c48f7960499d60fc6aa55/users/4db88167732f488faa3bfaeb77af6f27/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:48.525264 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb78dd-9d00-47a1-a5e9-01988e78219d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2d423182a3c48f7960499d60fc6aa55, user_id=4db88167732f488faa3bfaeb77af6f27, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:48.529780 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb78dd-9d00-47a1-a5e9-01988e78219d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:48.691353 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1153/2303] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:09:48 2022] PUT /identity/v3/projects/b2d423182a3c48f7960499d60fc6aa55/users/4db88167732f488faa3bfaeb77af6f27/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:48.705051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087a6a9-36ec-4889-8322-4afcc5f47035 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:48.705470 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087a6a9-36ec-4889-8322-4afcc5f47035 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:48.705835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0087a6a9-36ec-4889-8322-4afcc5f47035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:48.732845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-0087a6a9-36ec-4889-8322-4afcc5f47035 None None] MFA Rules not processed for user `4db88167732f488faa3bfaeb77af6f27`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:48.734062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0087a6a9-36ec-4889-8322-4afcc5f47035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:48.800114 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1151/2304] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:48.804789 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb78dd-9d00-47a1-a5e9-01988e78219d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:48.865448 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb78dd-9d00-47a1-a5e9-01988e78219d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:48.870197 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ad230-5f6f-457a-942d-9f4edc53364c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:48.870855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ad230-5f6f-457a-942d-9f4edc53364c admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:48.871127 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ad230-5f6f-457a-942d-9f4edc53364c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:48.871398 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ad230-5f6f-457a-942d-9f4edc53364c admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:48.872209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ad230-5f6f-457a-942d-9f4edc53364c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:48.874014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ad230-5f6f-457a-942d-9f4edc53364c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:48.995533 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1154/2305] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:09:48 2022] POST /identity/v3/projects => generated 379 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:09:48.999835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087a6a9-36ec-4889-8322-4afcc5f47035 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:49.013282 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0087a6a9-36ec-4889-8322-4afcc5f47035 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:49.053605 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:49.072152 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9405a4-01c2-49bc-a1de-254724967f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:49.072854 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9405a4-01c2-49bc-a1de-254724967f5f admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:49.073142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9405a4-01c2-49bc-a1de-254724967f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:49.073419 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9405a4-01c2-49bc-a1de-254724967f5f admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:49.074605 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9405a4-01c2-49bc-a1de-254724967f5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:49.076617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9405a4-01c2-49bc-a1de-254724967f5f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:49.153623 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1152/2306] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:09:48 2022] POST /identity/v3/users => generated 296 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:09:49.157724 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ad230-5f6f-457a-942d-9f4edc53364c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:49.168020 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ad230-5f6f-457a-942d-9f4edc53364c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:49.172724 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3dba6d-5252-4b1a-8426-8a71738cbd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:49.173372 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3dba6d-5252-4b1a-8426-8a71738cbd31 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:49.173652 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3dba6d-5252-4b1a-8426-8a71738cbd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:49.173925 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3dba6d-5252-4b1a-8426-8a71738cbd31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:49.174643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3dba6d-5252-4b1a-8426-8a71738cbd31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:49.176831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3dba6d-5252-4b1a-8426-8a71738cbd31 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:49.187446 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1155/2307] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:09:49.192283 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9405a4-01c2-49bc-a1de-254724967f5f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:49.202801 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9405a4-01c2-49bc-a1de-254724967f5f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:49.207716 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feccbfb7-111c-4ce2-af66-d12c1161c9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:49.208627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccbfb7-111c-4ce2-af66-d12c1161c9f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:49.208910 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccbfb7-111c-4ce2-af66-d12c1161c9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:49.209183 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feccbfb7-111c-4ce2-af66-d12c1161c9f3 admin admin] PATH_INFO: `/v3/projects/b9746b0b13b945baa5af49ea6684c810/users/509dac0716e34411b51a595a75764743/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:49.228458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feccbfb7-111c-4ce2-af66-d12c1161c9f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9746b0b13b945baa5af49ea6684c810, user_id=509dac0716e34411b51a595a75764743, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:49.228458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feccbfb7-111c-4ce2-af66-d12c1161c9f3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:49.349761 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1153/2308] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:09:49 2022] PUT /identity/v3/projects/b9746b0b13b945baa5af49ea6684c810/users/509dac0716e34411b51a595a75764743/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:49.353859 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3dba6d-5252-4b1a-8426-8a71738cbd31 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:49.364553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3dba6d-5252-4b1a-8426-8a71738cbd31 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:49.427840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb9bfe4-9b19-44a6-91b4-8f4613da5bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:49.428538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9bfe4-9b19-44a6-91b4-8f4613da5bce admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:49.428817 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9bfe4-9b19-44a6-91b4-8f4613da5bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:49.429090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9bfe4-9b19-44a6-91b4-8f4613da5bce admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:49.429808 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb9bfe4-9b19-44a6-91b4-8f4613da5bce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:49.431428 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb9bfe4-9b19-44a6-91b4-8f4613da5bce admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:49.441120 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1156/2309] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:09:49 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:09:49.445579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feccbfb7-111c-4ce2-af66-d12c1161c9f3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:49.456409 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feccbfb7-111c-4ce2-af66-d12c1161c9f3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:49.461338 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3dd33a-3d49-4324-b2d9-1a964e853bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:49.462210 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3dd33a-3d49-4324-b2d9-1a964e853bcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:49.462489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3dd33a-3d49-4324-b2d9-1a964e853bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:49.462762 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3dd33a-3d49-4324-b2d9-1a964e853bcf admin admin] PATH_INFO: `/v3/projects/b9746b0b13b945baa5af49ea6684c810/users/509dac0716e34411b51a595a75764743/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:49.466617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3dd33a-3d49-4324-b2d9-1a964e853bcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9746b0b13b945baa5af49ea6684c810, user_id=509dac0716e34411b51a595a75764743, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:49.468744 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3dd33a-3d49-4324-b2d9-1a964e853bcf admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:49.480698 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1154/2310] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:09:49 2022] PUT /identity/v3/projects/b9746b0b13b945baa5af49ea6684c810/users/509dac0716e34411b51a595a75764743/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:49.491531 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbcb00b-e2db-4e80-b4f8-d597f9a7de17 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:49.491826 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbcb00b-e2db-4e80-b4f8-d597f9a7de17 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:49.495337 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbcb00b-e2db-4e80-b4f8-d597f9a7de17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:49.518048 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-fbbcb00b-e2db-4e80-b4f8-d597f9a7de17 None None] MFA Rules not processed for user `509dac0716e34411b51a595a75764743`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:49.519257 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fbbcb00b-e2db-4e80-b4f8-d597f9a7de17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:49.548360 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:49.590035 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1157/2311] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:49 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:49.601361 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3dd33a-3d49-4324-b2d9-1a964e853bcf admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:49.603039 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-cf3dd33a-3d49-4324-b2d9-1a964e853bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:49.685152 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3dd33a-3d49-4324-b2d9-1a964e853bcf admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:49.689081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f3e88-1606-4db5-9cfd-fff391fff7e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:49.693237 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f3e88-1606-4db5-9cfd-fff391fff7e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:49.693237 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f3e88-1606-4db5-9cfd-fff391fff7e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:49.693237 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f3e88-1606-4db5-9cfd-fff391fff7e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:49.699928 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634f3e88-1606-4db5-9cfd-fff391fff7e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:49.701755 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634f3e88-1606-4db5-9cfd-fff391fff7e9 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:49.712178 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1155/2312] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:09:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:09:49.831631 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbcb00b-e2db-4e80-b4f8-d597f9a7de17 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:49.833285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fbbcb00b-e2db-4e80-b4f8-d597f9a7de17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:49.844726 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbcb00b-e2db-4e80-b4f8-d597f9a7de17 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:49.848801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b67efe-eb39-4a71-83ee-ec2aa709b043 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:49.849443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b67efe-eb39-4a71-83ee-ec2aa709b043 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:49.849719 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b67efe-eb39-4a71-83ee-ec2aa709b043 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:49.849986 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b67efe-eb39-4a71-83ee-ec2aa709b043 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:49.907138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b67efe-eb39-4a71-83ee-ec2aa709b043 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:49.908725 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b67efe-eb39-4a71-83ee-ec2aa709b043 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:49.922254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1158/2313] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:09:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:09:50.133966 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc1079-2d11-461f-ae7a-a6801051d00b None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:50.134266 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc1079-2d11-461f-ae7a-a6801051d00b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:50.134553 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc1079-2d11-461f-ae7a-a6801051d00b None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:50.135954 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1156/2314] 10.222.0.44 () {54 vars in 921 bytes} [Thu Jun 23 20:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:09:50.142106 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed430e87-2ac9-4e3b-b67f-435bc3e23264 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:50.142601 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed430e87-2ac9-4e3b-b67f-435bc3e23264 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:50.142689 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed430e87-2ac9-4e3b-b67f-435bc3e23264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:50.206649 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ed430e87-2ac9-4e3b-b67f-435bc3e23264 None None] MFA Rules not processed for user `a63d1d902c024c97a38f7d92f73814d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:50.208687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ed430e87-2ac9-4e3b-b67f-435bc3e23264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:50.314640 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1159/2315] 10.222.0.44 () {58 vars in 1018 bytes} [Thu Jun 23 20:09:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:50.328176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc1079-2d11-461f-ae7a-a6801051d00b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:50.329942 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-80dc1079-2d11-461f-ae7a-a6801051d00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:50.390183 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc1079-2d11-461f-ae7a-a6801051d00b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:50.393956 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4938d4a-43f2-43d1-9a6c-db24a481bca8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd72f952df34ad8853fe84673a41f45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:50.394644 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4938d4a-43f2-43d1-9a6c-db24a481bca8 service glance] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:50.395587 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4938d4a-43f2-43d1-9a6c-db24a481bca8 service glance] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:50.395861 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4938d4a-43f2-43d1-9a6c-db24a481bca8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:50.406435 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4938d4a-43f2-43d1-9a6c-db24a481bca8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:50.407936 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4938d4a-43f2-43d1-9a6c-db24a481bca8 service glance] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:50.418874 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1157/2316] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:09:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 20:09:50.641052 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448870b-0376-4b9e-b434-5809b44cc53e None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:50.641337 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448870b-0376-4b9e-b434-5809b44cc53e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:50.641627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448870b-0376-4b9e-b434-5809b44cc53e None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:50.643183 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1160/2317] 10.222.0.44 () {54 vars in 921 bytes} [Thu Jun 23 20:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:09:50.654015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761802b-ea9f-4be1-9f99-dbef1a92e90a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:50.654314 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761802b-ea9f-4be1-9f99-dbef1a92e90a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:50.654593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761802b-ea9f-4be1-9f99-dbef1a92e90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:50.673234 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e761802b-ea9f-4be1-9f99-dbef1a92e90a None None] MFA Rules not processed for user `332fbbefba114da4a01fdd496fff1e31`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:50.674499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e761802b-ea9f-4be1-9f99-dbef1a92e90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:50.687594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:50.727593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1158/2318] 10.222.0.44 () {58 vars in 1018 bytes} [Thu Jun 23 20:09:50 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:50.740328 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448870b-0376-4b9e-b434-5809b44cc53e None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:50.742001 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5448870b-0376-4b9e-b434-5809b44cc53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:50.755793 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448870b-0376-4b9e-b434-5809b44cc53e None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:50.759507 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a0970-58df-42fe-9be6-088fb4f20322 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:50.760141 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a0970-58df-42fe-9be6-088fb4f20322 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:50.760455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a0970-58df-42fe-9be6-088fb4f20322 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:50.760727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a0970-58df-42fe-9be6-088fb4f20322 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:50.772556 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a0970-58df-42fe-9be6-088fb4f20322 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:50.772556 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a0970-58df-42fe-9be6-088fb4f20322 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:50.777942 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1161/2319] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:09:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:09:50.860604 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e20b6-13fe-4666-87a6-d446e09ac3fc None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:50.862524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e20b6-13fe-4666-87a6-d446e09ac3fc None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:50.862524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e20b6-13fe-4666-87a6-d446e09ac3fc None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:50.862710 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1159/2320] 10.222.0.44 () {54 vars in 921 bytes} [Thu Jun 23 20:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:09:50.870104 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72337768-be7f-457f-b69e-2813bcf3d43e None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:50.870412 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72337768-be7f-457f-b69e-2813bcf3d43e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:50.870712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72337768-be7f-457f-b69e-2813bcf3d43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:50.877160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:50.889586 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-72337768-be7f-457f-b69e-2813bcf3d43e None None] MFA Rules not processed for user `a63d1d902c024c97a38f7d92f73814d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:50.890808 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-72337768-be7f-457f-b69e-2813bcf3d43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:50.900271 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1162/2321] 10.222.0.44 () {58 vars in 1018 bytes} [Thu Jun 23 20:09:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:50.911025 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e20b6-13fe-4666-87a6-d446e09ac3fc None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:50.921967 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e20b6-13fe-4666-87a6-d446e09ac3fc None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:50.925518 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c4151-2909-48d7-a81e-4cf14d6135e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:50.926730 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c4151-2909-48d7-a81e-4cf14d6135e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:50.927013 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c4151-2909-48d7-a81e-4cf14d6135e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:50.927284 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c4151-2909-48d7-a81e-4cf14d6135e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:50.937607 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3c4151-2909-48d7-a81e-4cf14d6135e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:50.939119 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3c4151-2909-48d7-a81e-4cf14d6135e2 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:50.950487 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1160/2322] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:09:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:09:57.110858 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95539fc3-a447-4374-a4dd-6d9429ba63ac None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:57.111311 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95539fc3-a447-4374-a4dd-6d9429ba63ac None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:57.111491 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95539fc3-a447-4374-a4dd-6d9429ba63ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:57.130060 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-95539fc3-a447-4374-a4dd-6d9429ba63ac None None] MFA Rules not processed for user `1d37f369749143ce813c693adc5f2ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:57.132413 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-95539fc3-a447-4374-a4dd-6d9429ba63ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:57.188726 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1163/2323] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jun 23 20:09:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:09:57.202248 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c4151-2909-48d7-a81e-4cf14d6135e2 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:57.202966 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-bc3c4151-2909-48d7-a81e-4cf14d6135e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:57.275822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c4151-2909-48d7-a81e-4cf14d6135e2 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:57.279130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc29a31-1dfa-41b4-ad5d-e60a6ce93600 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96db03079816461bb322633f64b9fb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:57.279767 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc29a31-1dfa-41b4-ad5d-e60a6ce93600 service swift] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:57.280047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc29a31-1dfa-41b4-ad5d-e60a6ce93600 service swift] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:57.280340 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc29a31-1dfa-41b4-ad5d-e60a6ce93600 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:57.288801 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc29a31-1dfa-41b4-ad5d-e60a6ce93600 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:57.290502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc29a31-1dfa-41b4-ad5d-e60a6ce93600 service swift] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:57.298754 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1161/2324] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jun 23 20:09:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 20:09:57.533846 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95539fc3-a447-4374-a4dd-6d9429ba63ac None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:57.604882 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95539fc3-a447-4374-a4dd-6d9429ba63ac None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:57.609972 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb62b0-bab4-4ecb-8698-c0428eb045f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:57.611042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb62b0-bab4-4ecb-8698-c0428eb045f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:57.611321 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb62b0-bab4-4ecb-8698-c0428eb045f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:57.611696 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb62b0-bab4-4ecb-8698-c0428eb045f1 admin admin] PATH_INFO: `/v3/users/1e69b6aceee244b890c0f623dbd8d390` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:57.612969 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fb62b0-bab4-4ecb-8698-c0428eb045f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e69b6aceee244b890c0f623dbd8d390)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:57.615141 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fb62b0-bab4-4ecb-8698-c0428eb045f1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:57.794731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-56fb62b0-bab4-4ecb-8698-c0428eb045f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e69b6aceee244b890c0f623dbd8d390'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:57.796202 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-56fb62b0-bab4-4ecb-8698-c0428eb045f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e69b6aceee244b890c0f623dbd8d390'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:57.873813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-56fb62b0-bab4-4ecb-8698-c0428eb045f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e69b6aceee244b890c0f623dbd8d390'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:57.874934 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-56fb62b0-bab4-4ecb-8698-c0428eb045f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e69b6aceee244b890c0f623dbd8d390'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:57.900896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-56fb62b0-bab4-4ecb-8698-c0428eb045f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e69b6aceee244b890c0f623dbd8d390'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:57.982161 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1164/2325] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:09:57 2022] DELETE /identity/v3/users/1e69b6aceee244b890c0f623dbd8d390 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:57.991063 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e7a63-d3b6-4a9e-be4b-e613706b9468 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:57.991063 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e7a63-d3b6-4a9e-be4b-e613706b9468 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:57.991063 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e7a63-d3b6-4a9e-be4b-e613706b9468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:58.017703 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9d3e7a63-d3b6-4a9e-be4b-e613706b9468 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:09:58.018934 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9d3e7a63-d3b6-4a9e-be4b-e613706b9468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:58.152580 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1162/2326] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:09:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:09:58.163643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb62b0-bab4-4ecb-8698-c0428eb045f1 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:58.165799 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-56fb62b0-bab4-4ecb-8698-c0428eb045f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:58.193414 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:58.244564 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb62b0-bab4-4ecb-8698-c0428eb045f1 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:58.249239 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09b647-1137-4084-b425-36710035d689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:58.250134 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09b647-1137-4084-b425-36710035d689 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:58.256111 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09b647-1137-4084-b425-36710035d689 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:58.256111 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09b647-1137-4084-b425-36710035d689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:58.262671 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a09b647-1137-4084-b425-36710035d689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:58.264139 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a09b647-1137-4084-b425-36710035d689 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:58.289198 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1165/2327] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:09:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:09:58.778815 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e7a63-d3b6-4a9e-be4b-e613706b9468 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:58.780540 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9d3e7a63-d3b6-4a9e-be4b-e613706b9468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:09:58.793628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e7a63-d3b6-4a9e-be4b-e613706b9468 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:58.801216 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b338d659-1a05-4849-9a30-2174022de60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:58.802552 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338d659-1a05-4849-9a30-2174022de60f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:58.802838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338d659-1a05-4849-9a30-2174022de60f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:58.804155 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338d659-1a05-4849-9a30-2174022de60f admin admin] PATH_INFO: `/v3/projects/6f02996c3b16439a80a20a126066b024` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:58.805113 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b338d659-1a05-4849-9a30-2174022de60f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f02996c3b16439a80a20a126066b024)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:58.807106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b338d659-1a05-4849-9a30-2174022de60f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:58.960901 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:09:58.974275 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b338d659-1a05-4849-9a30-2174022de60f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f02996c3b16439a80a20a126066b024', 'request_id': 'req-b338d659-1a05-4849-9a30-2174022de60f'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:58.983724 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b338d659-1a05-4849-9a30-2174022de60f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f02996c3b16439a80a20a126066b024', 'request_id': 'req-b338d659-1a05-4849-9a30-2174022de60f'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:58.990111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b338d659-1a05-4849-9a30-2174022de60f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f02996c3b16439a80a20a126066b024', 'request_id': 'req-b338d659-1a05-4849-9a30-2174022de60f'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:59.070372 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b338d659-1a05-4849-9a30-2174022de60f admin admin] The token cache is being invalidate because project 6f02996c3b16439a80a20a126066b024 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:09:59.070603 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b338d659-1a05-4849-9a30-2174022de60f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:59.071421 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b338d659-1a05-4849-9a30-2174022de60f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:59.073379 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1163/2328] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:09:58 2022] DELETE /identity/v3/projects/6f02996c3b16439a80a20a126066b024 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:09:59.078036 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09b647-1137-4084-b425-36710035d689 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:09:59.096495 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09b647-1137-4084-b425-36710035d689 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:09:59.215803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c48273-de51-4991-b5fc-10dbb842c6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:09:59.216818 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c48273-de51-4991-b5fc-10dbb842c6db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:09:59.217109 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c48273-de51-4991-b5fc-10dbb842c6db admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:09:59.217486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c48273-de51-4991-b5fc-10dbb842c6db admin admin] PATH_INFO: `/v3/users/4db88167732f488faa3bfaeb77af6f27` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:09:59.218883 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c48273-de51-4991-b5fc-10dbb842c6db admin admin] RBAC: Authorizing `identity:delete_user(user_id=4db88167732f488faa3bfaeb77af6f27)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:09:59.220883 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c48273-de51-4991-b5fc-10dbb842c6db admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:09:59.771870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-95c48273-de51-4991-b5fc-10dbb842c6db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4db88167732f488faa3bfaeb77af6f27'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:59.772707 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-95c48273-de51-4991-b5fc-10dbb842c6db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4db88167732f488faa3bfaeb77af6f27'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:59.845230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-95c48273-de51-4991-b5fc-10dbb842c6db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4db88167732f488faa3bfaeb77af6f27'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:59.846059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-95c48273-de51-4991-b5fc-10dbb842c6db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4db88167732f488faa3bfaeb77af6f27'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:59.884179 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-95c48273-de51-4991-b5fc-10dbb842c6db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4db88167732f488faa3bfaeb77af6f27'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:09:59.904827 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1166/2329] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:09:59 2022] DELETE /identity/v3/users/4db88167732f488faa3bfaeb77af6f27 => generated 0 bytes in 827 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:10:00.447948 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b338d659-1a05-4849-9a30-2174022de60f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:00.452033 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b338d659-1a05-4849-9a30-2174022de60f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:00.467324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b338d659-1a05-4849-9a30-2174022de60f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:00.546129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5936e-16b7-42ca-b862-b88f18c19c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:00.547179 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5936e-16b7-42ca-b862-b88f18c19c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:00.547470 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5936e-16b7-42ca-b862-b88f18c19c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:00.547748 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5936e-16b7-42ca-b862-b88f18c19c38 admin admin] PATH_INFO: `/v3/projects/b2d423182a3c48f7960499d60fc6aa55` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:00.550424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd5936e-16b7-42ca-b862-b88f18c19c38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2d423182a3c48f7960499d60fc6aa55)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:00.552888 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd5936e-16b7-42ca-b862-b88f18c19c38 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:00.711145 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-bcd5936e-16b7-42ca-b862-b88f18c19c38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2d423182a3c48f7960499d60fc6aa55', 'request_id': 'req-bcd5936e-16b7-42ca-b862-b88f18c19c38'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:00.723448 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-bcd5936e-16b7-42ca-b862-b88f18c19c38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2d423182a3c48f7960499d60fc6aa55', 'request_id': 'req-bcd5936e-16b7-42ca-b862-b88f18c19c38'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:00.732644 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-bcd5936e-16b7-42ca-b862-b88f18c19c38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2d423182a3c48f7960499d60fc6aa55', 'request_id': 'req-bcd5936e-16b7-42ca-b862-b88f18c19c38'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:00.812120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-bcd5936e-16b7-42ca-b862-b88f18c19c38 admin admin] The token cache is being invalidate because project b2d423182a3c48f7960499d60fc6aa55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:10:00.812581 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-bcd5936e-16b7-42ca-b862-b88f18c19c38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:00.813346 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-bcd5936e-16b7-42ca-b862-b88f18c19c38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:00.815122 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1164/2330] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:10:00 2022] DELETE /identity/v3/projects/b2d423182a3c48f7960499d60fc6aa55 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:10:00.819458 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c48273-de51-4991-b5fc-10dbb842c6db admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:00.821685 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-95c48273-de51-4991-b5fc-10dbb842c6db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:00.846812 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c48273-de51-4991-b5fc-10dbb842c6db admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:00.915213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:10:00.935933 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5277e5b-db74-42a1-b3d8-9348bee0cead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:00.936842 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5277e5b-db74-42a1-b3d8-9348bee0cead admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:00.937139 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5277e5b-db74-42a1-b3d8-9348bee0cead admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:00.937421 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5277e5b-db74-42a1-b3d8-9348bee0cead admin admin] PATH_INFO: `/v3/users/509dac0716e34411b51a595a75764743` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:00.938720 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5277e5b-db74-42a1-b3d8-9348bee0cead admin admin] RBAC: Authorizing `identity:delete_user(user_id=509dac0716e34411b51a595a75764743)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:00.940972 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5277e5b-db74-42a1-b3d8-9348bee0cead admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:01.182066 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a5277e5b-db74-42a1-b3d8-9348bee0cead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '509dac0716e34411b51a595a75764743'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:01.184163 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a5277e5b-db74-42a1-b3d8-9348bee0cead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '509dac0716e34411b51a595a75764743'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:01.314783 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a5277e5b-db74-42a1-b3d8-9348bee0cead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '509dac0716e34411b51a595a75764743'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:01.315884 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a5277e5b-db74-42a1-b3d8-9348bee0cead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '509dac0716e34411b51a595a75764743'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:01.348225 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a5277e5b-db74-42a1-b3d8-9348bee0cead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '509dac0716e34411b51a595a75764743'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:01.372703 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1167/2331] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:10:00 2022] DELETE /identity/v3/users/509dac0716e34411b51a595a75764743 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:10:02.221270 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acccbe7e-b6bd-42aa-9875-0c4586e7340b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:02.222672 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acccbe7e-b6bd-42aa-9875-0c4586e7340b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:02.223179 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acccbe7e-b6bd-42aa-9875-0c4586e7340b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:02.262957 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5277e5b-db74-42a1-b3d8-9348bee0cead admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:02.265226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a5277e5b-db74-42a1-b3d8-9348bee0cead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:02.327166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-acccbe7e-b6bd-42aa-9875-0c4586e7340b None None] MFA Rules not processed for user `1d37f369749143ce813c693adc5f2ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:10:02.329191 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-acccbe7e-b6bd-42aa-9875-0c4586e7340b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:02.383350 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:10:02.412598 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5277e5b-db74-42a1-b3d8-9348bee0cead admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:02.422514 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd97a33-129a-42f2-8e38-4a704de1160d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:02.423395 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd97a33-129a-42f2-8e38-4a704de1160d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:02.423683 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd97a33-129a-42f2-8e38-4a704de1160d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:02.423957 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd97a33-129a-42f2-8e38-4a704de1160d admin admin] PATH_INFO: `/v3/projects/b9746b0b13b945baa5af49ea6684c810` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:02.425318 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd97a33-129a-42f2-8e38-4a704de1160d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9746b0b13b945baa5af49ea6684c810)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:02.427447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd97a33-129a-42f2-8e38-4a704de1160d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:02.457432 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1165/2332] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jun 23 20:10:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:10:02.478556 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acccbe7e-b6bd-42aa-9875-0c4586e7340b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:02.481001 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-acccbe7e-b6bd-42aa-9875-0c4586e7340b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:02.585753 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6cd97a33-129a-42f2-8e38-4a704de1160d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9746b0b13b945baa5af49ea6684c810', 'request_id': 'req-6cd97a33-129a-42f2-8e38-4a704de1160d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:02.648323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acccbe7e-b6bd-42aa-9875-0c4586e7340b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:02.707340 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6cd97a33-129a-42f2-8e38-4a704de1160d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9746b0b13b945baa5af49ea6684c810', 'request_id': 'req-6cd97a33-129a-42f2-8e38-4a704de1160d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:02.717982 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:10:02.719918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6cd97a33-129a-42f2-8e38-4a704de1160d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9746b0b13b945baa5af49ea6684c810', 'request_id': 'req-6cd97a33-129a-42f2-8e38-4a704de1160d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:02.744485 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6cd97a33-129a-42f2-8e38-4a704de1160d admin admin] The token cache is being invalidate because project b9746b0b13b945baa5af49ea6684c810 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:10:02.745557 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6cd97a33-129a-42f2-8e38-4a704de1160d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:02.747405 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6cd97a33-129a-42f2-8e38-4a704de1160d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:02.751622 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1168/2333] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:10:02 2022] DELETE /identity/v3/projects/b9746b0b13b945baa5af49ea6684c810 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:10:02.762512 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b5e9b3-02ff-4c29-9753-5baaf0d492f8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96db03079816461bb322633f64b9fb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:02.763402 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5e9b3-02ff-4c29-9753-5baaf0d492f8 service swift] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:02.763867 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5e9b3-02ff-4c29-9753-5baaf0d492f8 service swift] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:02.764524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b5e9b3-02ff-4c29-9753-5baaf0d492f8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:02.768135 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-34b5e9b3-02ff-4c29-9753-5baaf0d492f8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:02.866538 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b5e9b3-02ff-4c29-9753-5baaf0d492f8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:02.868455 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b5e9b3-02ff-4c29-9753-5baaf0d492f8 service swift] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:02.888781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1166/2334] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jun 23 20:10:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 20:10:03.132411 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:10:13.427551 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:10:13.469297 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:10:14.404243 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3cdb4-9a55-4c0e-9f04-79f5460300d9 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:14.408221 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3cdb4-9a55-4c0e-9f04-79f5460300d9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:14.408221 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3cdb4-9a55-4c0e-9f04-79f5460300d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:14.422508 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-06b3cdb4-9a55-4c0e-9f04-79f5460300d9 None None] MFA Rules not processed for user `1d37f369749143ce813c693adc5f2ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:10:14.423809 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-06b3cdb4-9a55-4c0e-9f04-79f5460300d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:14.431526 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1169/2335] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jun 23 20:10:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:10:14.443376 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b5e9b3-02ff-4c29-9753-5baaf0d492f8 service swift] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:14.445388 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-34b5e9b3-02ff-4c29-9753-5baaf0d492f8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:14.458677 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b5e9b3-02ff-4c29-9753-5baaf0d492f8 service swift] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:14.463738 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56433a05-8a05-44ae-94af-1753e358472a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96db03079816461bb322633f64b9fb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:14.464529 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56433a05-8a05-44ae-94af-1753e358472a service swift] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:14.467370 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56433a05-8a05-44ae-94af-1753e358472a service swift] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:14.467370 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56433a05-8a05-44ae-94af-1753e358472a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:14.473009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56433a05-8a05-44ae-94af-1753e358472a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:14.474562 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56433a05-8a05-44ae-94af-1753e358472a service swift] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:14.482429 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1167/2336] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jun 23 20:10:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 20:10:15.208627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3594fe9-3f6d-4a69-b083-3f1ab4cc9406 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:15.209102 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3594fe9-3f6d-4a69-b083-3f1ab4cc9406 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:15.209333 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3594fe9-3f6d-4a69-b083-3f1ab4cc9406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:15.229133 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e3594fe9-3f6d-4a69-b083-3f1ab4cc9406 None None] MFA Rules not processed for user `1d37f369749143ce813c693adc5f2ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:10:15.231062 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e3594fe9-3f6d-4a69-b083-3f1ab4cc9406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:15.240455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1170/2337] 10.222.0.44 () {58 vars in 967 bytes} [Thu Jun 23 20:10:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:10:15.252640 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56433a05-8a05-44ae-94af-1753e358472a service swift] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:15.264493 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56433a05-8a05-44ae-94af-1753e358472a service swift] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:15.269826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0d4ba-e825-4918-a1c8-9c3fab46006d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96db03079816461bb322633f64b9fb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:15.270508 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0d4ba-e825-4918-a1c8-9c3fab46006d service swift] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:15.270740 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0d4ba-e825-4918-a1c8-9c3fab46006d service swift] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:15.271165 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0d4ba-e825-4918-a1c8-9c3fab46006d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:15.280527 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0d4ba-e825-4918-a1c8-9c3fab46006d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:15.282120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0d4ba-e825-4918-a1c8-9c3fab46006d service swift] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:15.290867 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1168/2338] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jun 23 20:10:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 20:10:15.509759 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be00f6-e633-4bda-a275-7ba103572534 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:15.510122 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be00f6-e633-4bda-a275-7ba103572534 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:15.510478 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be00f6-e633-4bda-a275-7ba103572534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:15.535436 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-62be00f6-e633-4bda-a275-7ba103572534 None None] MFA Rules not processed for user `1d37f369749143ce813c693adc5f2ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:10:15.536683 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-62be00f6-e633-4bda-a275-7ba103572534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:15.544563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1171/2339] 10.222.0.44 () {58 vars in 967 bytes} [Thu Jun 23 20:10:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:10:15.558699 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0d4ba-e825-4918-a1c8-9c3fab46006d service swift] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:15.569150 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0d4ba-e825-4918-a1c8-9c3fab46006d service swift] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:15.574270 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8befb-67a2-436e-9914-ba58bd977a09 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96db03079816461bb322633f64b9fb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:15.575161 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8befb-67a2-436e-9914-ba58bd977a09 service swift] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:15.575437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8befb-67a2-436e-9914-ba58bd977a09 service swift] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:15.575708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8befb-67a2-436e-9914-ba58bd977a09 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:15.584168 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8befb-67a2-436e-9914-ba58bd977a09 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:15.586171 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8befb-67a2-436e-9914-ba58bd977a09 service swift] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:15.594781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1169/2340] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jun 23 20:10:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 20:10:16.133630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be00f6-e633-4bda-a275-7ba103572534 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:16.135008 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-62be00f6-e633-4bda-a275-7ba103572534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:16.227248 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be00f6-e633-4bda-a275-7ba103572534 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:16.232411 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86457e7f-b010-4874-8183-a078e39bed18 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9715d5deeccd419b97cb0bd39e17982f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:16.233071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86457e7f-b010-4874-8183-a078e39bed18 service placement] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:16.233353 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86457e7f-b010-4874-8183-a078e39bed18 service placement] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:16.233629 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86457e7f-b010-4874-8183-a078e39bed18 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:16.235402 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-86457e7f-b010-4874-8183-a078e39bed18 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:16.251022 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86457e7f-b010-4874-8183-a078e39bed18 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:16.252729 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86457e7f-b010-4874-8183-a078e39bed18 service placement] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:16.262970 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1172/2341] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:10:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:10:16.302891 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c4f01-6d36-43b3-a56c-b354138a47e6 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:16.303221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c4f01-6d36-43b3-a56c-b354138a47e6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:16.303504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c4f01-6d36-43b3-a56c-b354138a47e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:16.321849 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-541c4f01-6d36-43b3-a56c-b354138a47e6 None None] MFA Rules not processed for user `1d37f369749143ce813c693adc5f2ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:10:16.323050 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-541c4f01-6d36-43b3-a56c-b354138a47e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:16.331222 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1170/2342] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jun 23 20:10:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:10:16.344387 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86457e7f-b010-4874-8183-a078e39bed18 service placement] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:16.355542 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86457e7f-b010-4874-8183-a078e39bed18 service placement] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:16.360711 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda560d-1dd4-49f0-a469-ae2ba05e44b9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96db03079816461bb322633f64b9fb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:16.361350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda560d-1dd4-49f0-a469-ae2ba05e44b9 service swift] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:16.361665 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda560d-1dd4-49f0-a469-ae2ba05e44b9 service swift] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:16.361942 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda560d-1dd4-49f0-a469-ae2ba05e44b9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:16.371771 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda560d-1dd4-49f0-a469-ae2ba05e44b9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:16.381729 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda560d-1dd4-49f0-a469-ae2ba05e44b9 service swift] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:16.382102 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1173/2343] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jun 23 20:10:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 20:10:16.962403 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0a231-c8f2-404b-ad1b-c500c5adc72d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:16.962877 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0a231-c8f2-404b-ad1b-c500c5adc72d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:16.962972 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0a231-c8f2-404b-ad1b-c500c5adc72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:16.980412 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-2aa0a231-c8f2-404b-ad1b-c500c5adc72d None None] MFA Rules not processed for user `1d37f369749143ce813c693adc5f2ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:10:16.981633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2aa0a231-c8f2-404b-ad1b-c500c5adc72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:16.988807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1171/2344] 10.222.0.44 () {58 vars in 967 bytes} [Thu Jun 23 20:10:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:10:17.001276 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda560d-1dd4-49f0-a469-ae2ba05e44b9 service swift] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:17.012442 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda560d-1dd4-49f0-a469-ae2ba05e44b9 service swift] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:17.017473 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ab1cb-547c-4b42-841c-b48fe821ed31 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96db03079816461bb322633f64b9fb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:17.018104 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ab1cb-547c-4b42-841c-b48fe821ed31 service swift] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:17.018438 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ab1cb-547c-4b42-841c-b48fe821ed31 service swift] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:17.018721 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ab1cb-547c-4b42-841c-b48fe821ed31 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:17.026683 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ab1cb-547c-4b42-841c-b48fe821ed31 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:17.028270 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ab1cb-547c-4b42-841c-b48fe821ed31 service swift] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:17.036221 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1174/2345] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jun 23 20:10:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 20:10:18.519514 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0a231-c8f2-404b-ad1b-c500c5adc72d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:18.520991 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2aa0a231-c8f2-404b-ad1b-c500c5adc72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:18.596535 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0a231-c8f2-404b-ad1b-c500c5adc72d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:18.601345 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51891556-7e59-4383-95b9-57cd3bdf1051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:18.602285 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51891556-7e59-4383-95b9-57cd3bdf1051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:18.602791 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51891556-7e59-4383-95b9-57cd3bdf1051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:18.603050 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51891556-7e59-4383-95b9-57cd3bdf1051 admin admin] PATH_INFO: `/v3/users/a6a6e5f6e5f24ba89ca4895aebe158f8` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:18.604899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51891556-7e59-4383-95b9-57cd3bdf1051 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6a6e5f6e5f24ba89ca4895aebe158f8)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:18.606937 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51891556-7e59-4383-95b9-57cd3bdf1051 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:18.853248 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-51891556-7e59-4383-95b9-57cd3bdf1051 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6a6e5f6e5f24ba89ca4895aebe158f8'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:18.853893 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-51891556-7e59-4383-95b9-57cd3bdf1051 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6a6e5f6e5f24ba89ca4895aebe158f8'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:18.865657 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-51891556-7e59-4383-95b9-57cd3bdf1051 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6a6e5f6e5f24ba89ca4895aebe158f8'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:18.872569 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-51891556-7e59-4383-95b9-57cd3bdf1051 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6a6e5f6e5f24ba89ca4895aebe158f8'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:18.948012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-51891556-7e59-4383-95b9-57cd3bdf1051 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6a6e5f6e5f24ba89ca4895aebe158f8'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:18.954415 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1172/2346] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:10:18 2022] DELETE /identity/v3/users/a6a6e5f6e5f24ba89ca4895aebe158f8 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:10:19.249548 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ab1cb-547c-4b42-841c-b48fe821ed31 service swift] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:19.259560 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ab1cb-547c-4b42-841c-b48fe821ed31 service swift] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:19.326344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19dec83-9fb9-4c19-97c3-eb43c4b3dcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:19.327211 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19dec83-9fb9-4c19-97c3-eb43c4b3dcdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:19.327480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19dec83-9fb9-4c19-97c3-eb43c4b3dcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:19.327752 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19dec83-9fb9-4c19-97c3-eb43c4b3dcdd admin admin] PATH_INFO: `/v3/projects/df7d693ea21441559f4db447f5414f69` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:19.328950 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19dec83-9fb9-4c19-97c3-eb43c4b3dcdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=df7d693ea21441559f4db447f5414f69)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:19.330895 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19dec83-9fb9-4c19-97c3-eb43c4b3dcdd admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:19.430405 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:10:19.447014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b19dec83-9fb9-4c19-97c3-eb43c4b3dcdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df7d693ea21441559f4db447f5414f69', 'request_id': 'req-b19dec83-9fb9-4c19-97c3-eb43c4b3dcdd'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:19.514613 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b19dec83-9fb9-4c19-97c3-eb43c4b3dcdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df7d693ea21441559f4db447f5414f69', 'request_id': 'req-b19dec83-9fb9-4c19-97c3-eb43c4b3dcdd'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:19.521264 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b19dec83-9fb9-4c19-97c3-eb43c4b3dcdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df7d693ea21441559f4db447f5414f69', 'request_id': 'req-b19dec83-9fb9-4c19-97c3-eb43c4b3dcdd'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:19.533993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b19dec83-9fb9-4c19-97c3-eb43c4b3dcdd admin admin] The token cache is being invalidate because project df7d693ea21441559f4db447f5414f69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:10:19.534277 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b19dec83-9fb9-4c19-97c3-eb43c4b3dcdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:19.535258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b19dec83-9fb9-4c19-97c3-eb43c4b3dcdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:19.537055 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1175/2347] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:10:19 2022] DELETE /identity/v3/projects/df7d693ea21441559f4db447f5414f69 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:10:19.544023 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd1d9c-08ef-4e00-a013-84d81bb40303 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:19.544554 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd1d9c-08ef-4e00-a013-84d81bb40303 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:19.546297 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd1d9c-08ef-4e00-a013-84d81bb40303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:19.563305 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7bfd1d9c-08ef-4e00-a013-84d81bb40303 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:10:19.564630 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7bfd1d9c-08ef-4e00-a013-84d81bb40303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:19.602223 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:10:19.634651 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1173/2348] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:10:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:10:19.642714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970962e0-aa77-41b7-bb19-e16cf1f8f592 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:19.642998 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970962e0-aa77-41b7-bb19-e16cf1f8f592 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:19.643271 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970962e0-aa77-41b7-bb19-e16cf1f8f592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:19.661194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-970962e0-aa77-41b7-bb19-e16cf1f8f592 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:10:19.662383 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-970962e0-aa77-41b7-bb19-e16cf1f8f592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:19.670815 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1176/2349] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:10:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:10:19.674988 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd1d9c-08ef-4e00-a013-84d81bb40303 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:19.685269 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd1d9c-08ef-4e00-a013-84d81bb40303 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:19.690264 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a12ee78-9857-4718-857f-1a339850a52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:19.690914 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12ee78-9857-4718-857f-1a339850a52a admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:19.691196 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12ee78-9857-4718-857f-1a339850a52a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:19.691473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12ee78-9857-4718-857f-1a339850a52a admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:19.692196 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a12ee78-9857-4718-857f-1a339850a52a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:10:19.692538 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a12ee78-9857-4718-857f-1a339850a52a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:19.694148 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a12ee78-9857-4718-857f-1a339850a52a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:19.706037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1174/2350] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:10:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:10:19.710970 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970962e0-aa77-41b7-bb19-e16cf1f8f592 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:19.720131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970962e0-aa77-41b7-bb19-e16cf1f8f592 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:19.724878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379edad0-80aa-4e7f-a302-eb7ed03e7644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:19.725516 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379edad0-80aa-4e7f-a302-eb7ed03e7644 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:19.725796 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379edad0-80aa-4e7f-a302-eb7ed03e7644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:19.726388 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379edad0-80aa-4e7f-a302-eb7ed03e7644 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:19.727085 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379edad0-80aa-4e7f-a302-eb7ed03e7644 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:19.729071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379edad0-80aa-4e7f-a302-eb7ed03e7644 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:19.837657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1177/2351] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:10:19 2022] POST /identity/v3/projects => generated 369 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:10:19.841695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a12ee78-9857-4718-857f-1a339850a52a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:19.851498 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379edad0-80aa-4e7f-a302-eb7ed03e7644 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:19.851880 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a12ee78-9857-4718-857f-1a339850a52a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:19.853297 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-379edad0-80aa-4e7f-a302-eb7ed03e7644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:19.909387 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50e2b9-49d5-4438-b532-31556d9eaf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:19.910147 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50e2b9-49d5-4438-b532-31556d9eaf67 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:19.910526 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50e2b9-49d5-4438-b532-31556d9eaf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:19.910799 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50e2b9-49d5-4438-b532-31556d9eaf67 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:19.911480 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e50e2b9-49d5-4438-b532-31556d9eaf67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:19.913494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e50e2b9-49d5-4438-b532-31556d9eaf67 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:19.917867 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379edad0-80aa-4e7f-a302-eb7ed03e7644 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:19.922663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85762b1b-9c01-4dac-a6cf-3d5654f49810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:19.923535 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85762b1b-9c01-4dac-a6cf-3d5654f49810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:19.923999 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85762b1b-9c01-4dac-a6cf-3d5654f49810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:19.924219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85762b1b-9c01-4dac-a6cf-3d5654f49810 admin admin] PATH_INFO: `/v3/users/3457dad643d34950834cc41462aa412b` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:19.925480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85762b1b-9c01-4dac-a6cf-3d5654f49810 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3457dad643d34950834cc41462aa412b)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:19.927413 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85762b1b-9c01-4dac-a6cf-3d5654f49810 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:20.023001 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1175/2352] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:10:19 2022] POST /identity/v3/users => generated 285 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:10:20.027464 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50e2b9-49d5-4438-b532-31556d9eaf67 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:20.036683 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50e2b9-49d5-4438-b532-31556d9eaf67 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:20.041945 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f341b84-5a42-43d7-bc9a-049d467f97b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:20.042573 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f341b84-5a42-43d7-bc9a-049d467f97b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:20.042850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f341b84-5a42-43d7-bc9a-049d467f97b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:20.043120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f341b84-5a42-43d7-bc9a-049d467f97b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:20.043894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f341b84-5a42-43d7-bc9a-049d467f97b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:20.045529 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f341b84-5a42-43d7-bc9a-049d467f97b2 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:20.056168 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1176/2353] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:10:20.060062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f341b84-5a42-43d7-bc9a-049d467f97b2 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:20.069536 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f341b84-5a42-43d7-bc9a-049d467f97b2 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:20.074203 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9d262-c1d4-4677-bfd5-e14f30c03524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:20.075066 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9d262-c1d4-4677-bfd5-e14f30c03524 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:20.078116 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9d262-c1d4-4677-bfd5-e14f30c03524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:20.078116 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9d262-c1d4-4677-bfd5-e14f30c03524 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:20.078116 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9d262-c1d4-4677-bfd5-e14f30c03524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:20.078116 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9d262-c1d4-4677-bfd5-e14f30c03524 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:20.088196 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1177/2354] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:10:20.091770 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9d262-c1d4-4677-bfd5-e14f30c03524 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:20.100792 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9d262-c1d4-4677-bfd5-e14f30c03524 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:20.106158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac3eed-696f-49f4-9c9b-05c08f10bf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:20.107012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac3eed-696f-49f4-9c9b-05c08f10bf25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:20.107316 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac3eed-696f-49f4-9c9b-05c08f10bf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:20.107624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac3eed-696f-49f4-9c9b-05c08f10bf25 admin admin] PATH_INFO: `/v3/projects/7b6689f3b35947c5b8ee3088a1f1fbde/users/a2f7959b30484321ba7ae5d2bbf8eb97/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:20.123393 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac3eed-696f-49f4-9c9b-05c08f10bf25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b6689f3b35947c5b8ee3088a1f1fbde, user_id=a2f7959b30484321ba7ae5d2bbf8eb97, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:20.126112 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac3eed-696f-49f4-9c9b-05c08f10bf25 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:20.141013 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-85762b1b-9c01-4dac-a6cf-3d5654f49810 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3457dad643d34950834cc41462aa412b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:20.141849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-85762b1b-9c01-4dac-a6cf-3d5654f49810 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3457dad643d34950834cc41462aa412b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:20.216956 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1178/2355] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:10:20 2022] PUT /identity/v3/projects/7b6689f3b35947c5b8ee3088a1f1fbde/users/a2f7959b30484321ba7ae5d2bbf8eb97/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:10:20.222471 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-85762b1b-9c01-4dac-a6cf-3d5654f49810 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3457dad643d34950834cc41462aa412b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:20.223165 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-85762b1b-9c01-4dac-a6cf-3d5654f49810 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3457dad643d34950834cc41462aa412b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:20.227241 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2499f6-57e3-4178-8393-f0e231b65fc0 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:20.227535 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2499f6-57e3-4178-8393-f0e231b65fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:20.227810 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2499f6-57e3-4178-8393-f0e231b65fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:20.237392 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-85762b1b-9c01-4dac-a6cf-3d5654f49810 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3457dad643d34950834cc41462aa412b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:20.251215 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1178/2356] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:10:19 2022] DELETE /identity/v3/users/3457dad643d34950834cc41462aa412b => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:10:20.255627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cba290-711d-42f3-8bb5-67d0d1400fff None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:20.255757 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4f2499f6-57e3-4178-8393-f0e231b65fc0 None None] MFA Rules not processed for user `a2f7959b30484321ba7ae5d2bbf8eb97`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:10:20.255917 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cba290-711d-42f3-8bb5-67d0d1400fff None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:20.256196 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cba290-711d-42f3-8bb5-67d0d1400fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:20.256880 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4f2499f6-57e3-4178-8393-f0e231b65fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:20.262735 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:10:20.273239 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e3cba290-711d-42f3-8bb5-67d0d1400fff None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:10:20.274485 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e3cba290-711d-42f3-8bb5-67d0d1400fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:20.306799 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:10:20.330302 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1179/2357] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:10:20 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:10:20.342022 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2499f6-57e3-4178-8393-f0e231b65fc0 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:20.343274 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4f2499f6-57e3-4178-8393-f0e231b65fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:20.343912 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1179/2358] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:10:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:10:20.356292 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cba290-711d-42f3-8bb5-67d0d1400fff None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:20.357812 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e3cba290-711d-42f3-8bb5-67d0d1400fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:20.412727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cba290-711d-42f3-8bb5-67d0d1400fff None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:20.417174 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701426c7-c724-4d0c-b5dd-4810ec0112be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:20.417929 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701426c7-c724-4d0c-b5dd-4810ec0112be service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:20.418291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701426c7-c724-4d0c-b5dd-4810ec0112be service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:20.418665 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701426c7-c724-4d0c-b5dd-4810ec0112be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:20.424038 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2499f6-57e3-4178-8393-f0e231b65fc0 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:20.427838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701426c7-c724-4d0c-b5dd-4810ec0112be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:20.428246 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813db14-5a9a-459f-8db1-75baabfaeae3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd72f952df34ad8853fe84673a41f45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:20.429155 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813db14-5a9a-459f-8db1-75baabfaeae3 service glance] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:20.432660 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701426c7-c724-4d0c-b5dd-4810ec0112be service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:20.432791 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813db14-5a9a-459f-8db1-75baabfaeae3 service glance] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:20.432791 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813db14-5a9a-459f-8db1-75baabfaeae3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:20.438107 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c813db14-5a9a-459f-8db1-75baabfaeae3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:20.439272 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1180/2359] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:10:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:10:20.439593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c813db14-5a9a-459f-8db1-75baabfaeae3 service glance] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:20.448987 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1180/2360] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:10:20 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 20:10:20.708538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701426c7-c724-4d0c-b5dd-4810ec0112be service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:20.713534 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-701426c7-c724-4d0c-b5dd-4810ec0112be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:20.725412 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701426c7-c724-4d0c-b5dd-4810ec0112be service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:20.738048 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb2931e-cac7-43d3-82ad-5600908ab344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:20.739327 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2931e-cac7-43d3-82ad-5600908ab344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:20.739795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2931e-cac7-43d3-82ad-5600908ab344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:20.740320 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2931e-cac7-43d3-82ad-5600908ab344 admin admin] PATH_INFO: `/v3/projects/fee2271744814fe9b2726faa89cfe1bc` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:20.742315 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb2931e-cac7-43d3-82ad-5600908ab344 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fee2271744814fe9b2726faa89cfe1bc)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:20.744615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb2931e-cac7-43d3-82ad-5600908ab344 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:20.874685 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-efb2931e-cac7-43d3-82ad-5600908ab344 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fee2271744814fe9b2726faa89cfe1bc', 'request_id': 'req-efb2931e-cac7-43d3-82ad-5600908ab344'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:20.958888 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-efb2931e-cac7-43d3-82ad-5600908ab344 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fee2271744814fe9b2726faa89cfe1bc', 'request_id': 'req-efb2931e-cac7-43d3-82ad-5600908ab344'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:20.965649 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-efb2931e-cac7-43d3-82ad-5600908ab344 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fee2271744814fe9b2726faa89cfe1bc', 'request_id': 'req-efb2931e-cac7-43d3-82ad-5600908ab344'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:20.976949 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-efb2931e-cac7-43d3-82ad-5600908ab344 admin admin] The token cache is being invalidate because project fee2271744814fe9b2726faa89cfe1bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:10:20.977228 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-efb2931e-cac7-43d3-82ad-5600908ab344 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:20.977927 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-efb2931e-cac7-43d3-82ad-5600908ab344 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:20.979572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1181/2361] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:10:20 2022] DELETE /identity/v3/projects/fee2271744814fe9b2726faa89cfe1bc => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:10:20.984496 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b806f8-c12c-4b8a-b914-600f4d40c4b1 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:20.984960 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b806f8-c12c-4b8a-b914-600f4d40c4b1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:20.985075 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b806f8-c12c-4b8a-b914-600f4d40c4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:21.003711 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-92b806f8-c12c-4b8a-b914-600f4d40c4b1 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:10:21.004935 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-92b806f8-c12c-4b8a-b914-600f4d40c4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:21.068685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1181/2362] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:10:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:10:21.077501 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626a11a-5816-45b0-8dbe-600407c32874 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:21.077796 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626a11a-5816-45b0-8dbe-600407c32874 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:21.078073 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626a11a-5816-45b0-8dbe-600407c32874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:21.096624 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3626a11a-5816-45b0-8dbe-600407c32874 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:10:21.097830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3626a11a-5816-45b0-8dbe-600407c32874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:21.105734 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1182/2363] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:10:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:10:21.110082 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b806f8-c12c-4b8a-b914-600f4d40c4b1 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:21.119732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b806f8-c12c-4b8a-b914-600f4d40c4b1 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:21.124577 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f2236-7132-49fa-ad81-5419b13f31b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:21.125310 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f2236-7132-49fa-ad81-5419b13f31b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:21.125544 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f2236-7132-49fa-ad81-5419b13f31b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:21.125815 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f2236-7132-49fa-ad81-5419b13f31b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:21.126540 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f2236-7132-49fa-ad81-5419b13f31b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:10:21.126845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f2236-7132-49fa-ad81-5419b13f31b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:21.128514 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f2236-7132-49fa-ad81-5419b13f31b7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:21.141929 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1182/2364] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:10:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:10:21.146881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626a11a-5816-45b0-8dbe-600407c32874 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:21.156990 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626a11a-5816-45b0-8dbe-600407c32874 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:21.161806 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9df9af-85b7-483b-92ae-0c52090beda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:21.162434 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9df9af-85b7-483b-92ae-0c52090beda5 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:21.162716 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9df9af-85b7-483b-92ae-0c52090beda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:21.162990 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9df9af-85b7-483b-92ae-0c52090beda5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:21.163707 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9df9af-85b7-483b-92ae-0c52090beda5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:21.169202 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9df9af-85b7-483b-92ae-0c52090beda5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:21.254215 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1183/2365] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:10:21 2022] POST /identity/v3/projects => generated 399 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:10:21.259645 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f2236-7132-49fa-ad81-5419b13f31b7 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:21.270830 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f2236-7132-49fa-ad81-5419b13f31b7 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:21.334058 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:10:21.340013 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb67fe6-8f6d-466c-a828-3c038542a21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:21.340810 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb67fe6-8f6d-466c-a828-3c038542a21f admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:21.341277 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb67fe6-8f6d-466c-a828-3c038542a21f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:21.341558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb67fe6-8f6d-466c-a828-3c038542a21f admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:21.342258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb67fe6-8f6d-466c-a828-3c038542a21f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:21.344596 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb67fe6-8f6d-466c-a828-3c038542a21f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:21.411378 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1183/2366] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:10:21 2022] POST /identity/v3/users => generated 300 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:10:21.415499 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9df9af-85b7-483b-92ae-0c52090beda5 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:21.421050 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:10:21.426225 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9df9af-85b7-483b-92ae-0c52090beda5 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:21.431258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbc75d-d046-489f-b1d0-7bb739c91b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:21.432040 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbc75d-d046-489f-b1d0-7bb739c91b75 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:21.432257 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbc75d-d046-489f-b1d0-7bb739c91b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:21.432568 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bbc75d-d046-489f-b1d0-7bb739c91b75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:21.433290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bbc75d-d046-489f-b1d0-7bb739c91b75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:21.434897 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bbc75d-d046-489f-b1d0-7bb739c91b75 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:21.446527 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1184/2367] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:10:21.450989 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb67fe6-8f6d-466c-a828-3c038542a21f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:21.461477 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb67fe6-8f6d-466c-a828-3c038542a21f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:21.466163 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ddfd2-74d5-47cc-bcd0-f5ec86c283ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:21.466832 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ddfd2-74d5-47cc-bcd0-f5ec86c283ed admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:21.467106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ddfd2-74d5-47cc-bcd0-f5ec86c283ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:21.467382 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ddfd2-74d5-47cc-bcd0-f5ec86c283ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:21.468111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7ddfd2-74d5-47cc-bcd0-f5ec86c283ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:21.471098 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7ddfd2-74d5-47cc-bcd0-f5ec86c283ed admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:21.479843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1184/2368] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:10:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:10:21.483933 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbc75d-d046-489f-b1d0-7bb739c91b75 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:21.494674 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bbc75d-d046-489f-b1d0-7bb739c91b75 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:21.499344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc05bf1-8daa-4e52-81cd-9d900988da0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:21.508575 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc05bf1-8daa-4e52-81cd-9d900988da0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:21.508575 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc05bf1-8daa-4e52-81cd-9d900988da0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:21.508575 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc05bf1-8daa-4e52-81cd-9d900988da0c admin admin] PATH_INFO: `/v3/projects/d9a16c22a8e84b418532658ba0e82bdc/users/63ba7a297e0543b09da4995f088a1f9e/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:21.515630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc05bf1-8daa-4e52-81cd-9d900988da0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9a16c22a8e84b418532658ba0e82bdc, user_id=63ba7a297e0543b09da4995f088a1f9e, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:21.518363 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc05bf1-8daa-4e52-81cd-9d900988da0c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:21.600073 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1185/2369] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:10:21 2022] PUT /identity/v3/projects/d9a16c22a8e84b418532658ba0e82bdc/users/63ba7a297e0543b09da4995f088a1f9e/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:10:21.610944 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee184179-0700-4a46-9cb0-20a7c8fade0a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:21.611235 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee184179-0700-4a46-9cb0-20a7c8fade0a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:21.611516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee184179-0700-4a46-9cb0-20a7c8fade0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:21.637992 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ee184179-0700-4a46-9cb0-20a7c8fade0a None None] MFA Rules not processed for user `63ba7a297e0543b09da4995f088a1f9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:10:21.639170 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ee184179-0700-4a46-9cb0-20a7c8fade0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:21.703479 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1185/2370] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:10:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:10:21.715716 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc05bf1-8daa-4e52-81cd-9d900988da0c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:21.717601 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3fc05bf1-8daa-4e52-81cd-9d900988da0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:21.789254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc05bf1-8daa-4e52-81cd-9d900988da0c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:21.793438 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d533c-d3ad-49b6-a452-764304a1ce32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:21.794082 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d533c-d3ad-49b6-a452-764304a1ce32 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:21.794359 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d533c-d3ad-49b6-a452-764304a1ce32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:21.795038 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d533c-d3ad-49b6-a452-764304a1ce32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:21.803690 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73d533c-d3ad-49b6-a452-764304a1ce32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:21.805921 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73d533c-d3ad-49b6-a452-764304a1ce32 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:21.816087 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1186/2371] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:10:21 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:10:30.179475 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041bb016-d083-4e08-b6a6-144fecfbd6c1 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:30.179980 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041bb016-d083-4e08-b6a6-144fecfbd6c1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:30.180105 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041bb016-d083-4e08-b6a6-144fecfbd6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:30.199698 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-041bb016-d083-4e08-b6a6-144fecfbd6c1 None None] MFA Rules not processed for user `1d37f369749143ce813c693adc5f2ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:10:30.201320 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-041bb016-d083-4e08-b6a6-144fecfbd6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:30.272387 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1186/2372] 10.222.0.44 () {58 vars in 967 bytes} [Thu Jun 23 20:10:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:10:30.296160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d533c-d3ad-49b6-a452-764304a1ce32 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:30.297650 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a73d533c-d3ad-49b6-a452-764304a1ce32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:30.381034 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d533c-d3ad-49b6-a452-764304a1ce32 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:30.387950 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90bd9c-572d-44ab-a6e7-8bd247f3c3fc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96db03079816461bb322633f64b9fb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:30.388693 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90bd9c-572d-44ab-a6e7-8bd247f3c3fc service swift] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:30.388947 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90bd9c-572d-44ab-a6e7-8bd247f3c3fc service swift] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:30.389226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90bd9c-572d-44ab-a6e7-8bd247f3c3fc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:30.398820 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd90bd9c-572d-44ab-a6e7-8bd247f3c3fc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:30.402508 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd90bd9c-572d-44ab-a6e7-8bd247f3c3fc service swift] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:30.411304 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1187/2373] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jun 23 20:10:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 20:10:30.781601 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041bb016-d083-4e08-b6a6-144fecfbd6c1 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:30.848248 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041bb016-d083-4e08-b6a6-144fecfbd6c1 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:30.853450 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e44e0-73b4-450d-9f0d-e178c60d60da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:30.854323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e44e0-73b4-450d-9f0d-e178c60d60da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:30.854599 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e44e0-73b4-450d-9f0d-e178c60d60da admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:30.854869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e44e0-73b4-450d-9f0d-e178c60d60da admin admin] PATH_INFO: `/v3/users/63ba7a297e0543b09da4995f088a1f9e` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:30.856162 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e44e0-73b4-450d-9f0d-e178c60d60da admin admin] RBAC: Authorizing `identity:delete_user(user_id=63ba7a297e0543b09da4995f088a1f9e)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:30.858168 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e44e0-73b4-450d-9f0d-e178c60d60da admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:30.871109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:10:31.064738 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c78e44e0-73b4-450d-9f0d-e178c60d60da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63ba7a297e0543b09da4995f088a1f9e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:31.065643 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c78e44e0-73b4-450d-9f0d-e178c60d60da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63ba7a297e0543b09da4995f088a1f9e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:31.078373 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c78e44e0-73b4-450d-9f0d-e178c60d60da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63ba7a297e0543b09da4995f088a1f9e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:31.085449 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c78e44e0-73b4-450d-9f0d-e178c60d60da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63ba7a297e0543b09da4995f088a1f9e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:31.199356 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c78e44e0-73b4-450d-9f0d-e178c60d60da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63ba7a297e0543b09da4995f088a1f9e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:31.206764 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1187/2374] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:10:30 2022] DELETE /identity/v3/users/63ba7a297e0543b09da4995f088a1f9e => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:10:31.210920 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4147a-b3e8-4897-9bea-a432f2bf5900 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:31.211495 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4147a-b3e8-4897-9bea-a432f2bf5900 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:31.211782 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4147a-b3e8-4897-9bea-a432f2bf5900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:31.233160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f4e4147a-b3e8-4897-9bea-a432f2bf5900 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:10:31.234580 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f4e4147a-b3e8-4897-9bea-a432f2bf5900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:31.293432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:10:31.301767 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1188/2375] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:10:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:10:31.312084 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e44e0-73b4-450d-9f0d-e178c60d60da admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:31.313712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c78e44e0-73b4-450d-9f0d-e178c60d60da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:31.376498 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e44e0-73b4-450d-9f0d-e178c60d60da admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:31.380184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ab7ae-d7d3-42ba-b586-365384f8bb2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:31.380859 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ab7ae-d7d3-42ba-b586-365384f8bb2e service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:31.381146 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ab7ae-d7d3-42ba-b586-365384f8bb2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:31.381441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ab7ae-d7d3-42ba-b586-365384f8bb2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:31.391092 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625ab7ae-d7d3-42ba-b586-365384f8bb2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:31.392838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625ab7ae-d7d3-42ba-b586-365384f8bb2e service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:31.403453 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1188/2376] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:10:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:10:32.247544 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4147a-b3e8-4897-9bea-a432f2bf5900 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:32.249153 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f4e4147a-b3e8-4897-9bea-a432f2bf5900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:32.280743 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4147a-b3e8-4897-9bea-a432f2bf5900 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:32.299568 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbcb792-b2f5-4275-a3c7-19edfef008f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:32.300492 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbcb792-b2f5-4275-a3c7-19edfef008f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:32.300819 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbcb792-b2f5-4275-a3c7-19edfef008f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:32.301060 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbcb792-b2f5-4275-a3c7-19edfef008f0 admin admin] PATH_INFO: `/v3/projects/d9a16c22a8e84b418532658ba0e82bdc` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:32.302319 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbcb792-b2f5-4275-a3c7-19edfef008f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9a16c22a8e84b418532658ba0e82bdc)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:32.305068 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbcb792-b2f5-4275-a3c7-19edfef008f0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:32.437378 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5cbcb792-b2f5-4275-a3c7-19edfef008f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9a16c22a8e84b418532658ba0e82bdc', 'request_id': 'req-5cbcb792-b2f5-4275-a3c7-19edfef008f0'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:32.537241 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5cbcb792-b2f5-4275-a3c7-19edfef008f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9a16c22a8e84b418532658ba0e82bdc', 'request_id': 'req-5cbcb792-b2f5-4275-a3c7-19edfef008f0'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:32.545519 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5cbcb792-b2f5-4275-a3c7-19edfef008f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9a16c22a8e84b418532658ba0e82bdc', 'request_id': 'req-5cbcb792-b2f5-4275-a3c7-19edfef008f0'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:32.559476 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5cbcb792-b2f5-4275-a3c7-19edfef008f0 admin admin] The token cache is being invalidate because project d9a16c22a8e84b418532658ba0e82bdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:10:32.559786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5cbcb792-b2f5-4275-a3c7-19edfef008f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:32.560620 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5cbcb792-b2f5-4275-a3c7-19edfef008f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:32.568861 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1189/2377] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:10:32 2022] DELETE /identity/v3/projects/d9a16c22a8e84b418532658ba0e82bdc => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:10:41.398865 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=48,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:10:42.555225 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=27,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:10:48.344400 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42380bc8-6314-4ac2-b7e6-470626036577 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:48.350807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42380bc8-6314-4ac2-b7e6-470626036577 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:48.350807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42380bc8-6314-4ac2-b7e6-470626036577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:48.362845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-42380bc8-6314-4ac2-b7e6-470626036577 None None] MFA Rules not processed for user `8a5b19c8c0d44e1f93bb1dea72884cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:10:48.364039 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-42380bc8-6314-4ac2-b7e6-470626036577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:48.419412 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1189/2378] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:10:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:10:53.156715 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a59a9-d612-423a-989d-0be07edfc96b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:53.157242 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a59a9-d612-423a-989d-0be07edfc96b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:53.157336 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a59a9-d612-423a-989d-0be07edfc96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:53.175193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-074a59a9-d612-423a-989d-0be07edfc96b None None] MFA Rules not processed for user `f041449191c94a5ca5a38c29dd146a15`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:10:53.176440 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-074a59a9-d612-423a-989d-0be07edfc96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:53.235094 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1190/2379] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:10:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:10:55.783459 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42380bc8-6314-4ac2-b7e6-470626036577 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:55.784860 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-42380bc8-6314-4ac2-b7e6-470626036577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:55.850561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42380bc8-6314-4ac2-b7e6-470626036577 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:55.855612 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a78083-0a39-468d-92ec-57b67333f589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:55.856512 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a78083-0a39-468d-92ec-57b67333f589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:55.856818 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a78083-0a39-468d-92ec-57b67333f589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:55.857092 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a78083-0a39-468d-92ec-57b67333f589 admin admin] PATH_INFO: `/v3/users/a2f7959b30484321ba7ae5d2bbf8eb97` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:55.858292 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a78083-0a39-468d-92ec-57b67333f589 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2f7959b30484321ba7ae5d2bbf8eb97)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:55.860224 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a78083-0a39-468d-92ec-57b67333f589 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:56.066983 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e3a78083-0a39-468d-92ec-57b67333f589 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2f7959b30484321ba7ae5d2bbf8eb97'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:56.067678 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e3a78083-0a39-468d-92ec-57b67333f589 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2f7959b30484321ba7ae5d2bbf8eb97'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:56.081593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e3a78083-0a39-468d-92ec-57b67333f589 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2f7959b30484321ba7ae5d2bbf8eb97'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:56.089954 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e3a78083-0a39-468d-92ec-57b67333f589 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2f7959b30484321ba7ae5d2bbf8eb97'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:56.182596 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e3a78083-0a39-468d-92ec-57b67333f589 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2f7959b30484321ba7ae5d2bbf8eb97'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:56.190025 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1190/2380] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:10:55 2022] DELETE /identity/v3/users/a2f7959b30484321ba7ae5d2bbf8eb97 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:10:56.194851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8c338-1d48-4cd6-a03d-28088fcd87bc None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:56.195397 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8c338-1d48-4cd6-a03d-28088fcd87bc None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:56.195397 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8c338-1d48-4cd6-a03d-28088fcd87bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:56.215157 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-78c8c338-1d48-4cd6-a03d-28088fcd87bc None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:10:56.216483 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-78c8c338-1d48-4cd6-a03d-28088fcd87bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:56.287436 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1191/2381] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:10:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:10:56.294883 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a78083-0a39-468d-92ec-57b67333f589 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:56.296536 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e3a78083-0a39-468d-92ec-57b67333f589 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:56.327827 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:10:56.359907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a78083-0a39-468d-92ec-57b67333f589 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:56.363467 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa54d34-4f82-4b06-b153-94c5d41732cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:56.364125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa54d34-4f82-4b06-b153-94c5d41732cb service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:56.364447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa54d34-4f82-4b06-b153-94c5d41732cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:56.367557 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa54d34-4f82-4b06-b153-94c5d41732cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:56.374458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa54d34-4f82-4b06-b153-94c5d41732cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:56.375900 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa54d34-4f82-4b06-b153-94c5d41732cb service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:56.386091 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1191/2382] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:10:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:10:56.794969 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8c338-1d48-4cd6-a03d-28088fcd87bc None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:56.796622 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-78c8c338-1d48-4cd6-a03d-28088fcd87bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:56.810533 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8c338-1d48-4cd6-a03d-28088fcd87bc None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:56.816817 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7fef0e-4050-489a-b349-82822bcfd0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:56.817347 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fef0e-4050-489a-b349-82822bcfd0d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:56.817623 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fef0e-4050-489a-b349-82822bcfd0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:56.817902 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fef0e-4050-489a-b349-82822bcfd0d2 admin admin] PATH_INFO: `/v3/projects/7b6689f3b35947c5b8ee3088a1f1fbde` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:56.819122 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7fef0e-4050-489a-b349-82822bcfd0d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b6689f3b35947c5b8ee3088a1f1fbde)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:56.821312 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7fef0e-4050-489a-b349-82822bcfd0d2 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:56.957225 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dc7fef0e-4050-489a-b349-82822bcfd0d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b6689f3b35947c5b8ee3088a1f1fbde', 'request_id': 'req-dc7fef0e-4050-489a-b349-82822bcfd0d2'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:57.033176 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dc7fef0e-4050-489a-b349-82822bcfd0d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b6689f3b35947c5b8ee3088a1f1fbde', 'request_id': 'req-dc7fef0e-4050-489a-b349-82822bcfd0d2'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:57.039287 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dc7fef0e-4050-489a-b349-82822bcfd0d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b6689f3b35947c5b8ee3088a1f1fbde', 'request_id': 'req-dc7fef0e-4050-489a-b349-82822bcfd0d2'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:57.051087 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dc7fef0e-4050-489a-b349-82822bcfd0d2 admin admin] The token cache is being invalidate because project 7b6689f3b35947c5b8ee3088a1f1fbde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:10:57.051379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dc7fef0e-4050-489a-b349-82822bcfd0d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:57.052023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-dc7fef0e-4050-489a-b349-82822bcfd0d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:57.054048 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1192/2383] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:10:56 2022] DELETE /identity/v3/projects/7b6689f3b35947c5b8ee3088a1f1fbde => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:10:57.059334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d6a7f-97a9-48a1-b0e6-2689f6aaaf06 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:57.059802 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d6a7f-97a9-48a1-b0e6-2689f6aaaf06 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:57.059910 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d6a7f-97a9-48a1-b0e6-2689f6aaaf06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:57.078928 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-326d6a7f-97a9-48a1-b0e6-2689f6aaaf06 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:10:57.080243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-326d6a7f-97a9-48a1-b0e6-2689f6aaaf06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:57.140739 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1192/2384] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:10:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:10:57.149400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c294b77-0e15-4720-ae1a-71fdb9615242 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:57.149754 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c294b77-0e15-4720-ae1a-71fdb9615242 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:57.150095 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c294b77-0e15-4720-ae1a-71fdb9615242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:57.170159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-1c294b77-0e15-4720-ae1a-71fdb9615242 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:10:57.171457 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1c294b77-0e15-4720-ae1a-71fdb9615242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:57.179501 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1193/2385] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:10:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:10:57.183653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d6a7f-97a9-48a1-b0e6-2689f6aaaf06 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:57.193130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d6a7f-97a9-48a1-b0e6-2689f6aaaf06 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:57.198122 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463c63b-1495-42c6-9695-2787d367ef89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:57.198767 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463c63b-1495-42c6-9695-2787d367ef89 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:57.199045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463c63b-1495-42c6-9695-2787d367ef89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:57.199314 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463c63b-1495-42c6-9695-2787d367ef89 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:57.200033 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1463c63b-1495-42c6-9695-2787d367ef89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:10:57.200368 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1463c63b-1495-42c6-9695-2787d367ef89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:57.202259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1463c63b-1495-42c6-9695-2787d367ef89 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:57.218062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1193/2386] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:10:57.219522 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c294b77-0e15-4720-ae1a-71fdb9615242 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:57.229872 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c294b77-0e15-4720-ae1a-71fdb9615242 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:57.241012 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8a0d6-07db-4b41-8442-1aabd9db95fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:57.241677 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8a0d6-07db-4b41-8442-1aabd9db95fc admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:57.241956 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8a0d6-07db-4b41-8442-1aabd9db95fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:57.242225 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8a0d6-07db-4b41-8442-1aabd9db95fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:57.242945 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb8a0d6-07db-4b41-8442-1aabd9db95fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:57.245083 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb8a0d6-07db-4b41-8442-1aabd9db95fc admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:57.383863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:10:57.390239 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1194/2387] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:10:57 2022] POST /identity/v3/projects => generated 371 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:10:57.394602 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463c63b-1495-42c6-9695-2787d367ef89 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:57.405554 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463c63b-1495-42c6-9695-2787d367ef89 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:57.468741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4594d-11e6-486a-aa46-0945ba71267c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:57.469422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4594d-11e6-486a-aa46-0945ba71267c admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:57.469705 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4594d-11e6-486a-aa46-0945ba71267c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:57.469977 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c4594d-11e6-486a-aa46-0945ba71267c admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:57.470672 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c4594d-11e6-486a-aa46-0945ba71267c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:57.472667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c4594d-11e6-486a-aa46-0945ba71267c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:57.620436 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1194/2388] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:10:57 2022] POST /identity/v3/users => generated 286 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:10:57.621546 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8a0d6-07db-4b41-8442-1aabd9db95fc admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:57.631592 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8a0d6-07db-4b41-8442-1aabd9db95fc admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:57.636656 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ac481-8a28-42eb-9c34-c7d8753407ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:57.637307 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ac481-8a28-42eb-9c34-c7d8753407ce admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:57.637597 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ac481-8a28-42eb-9c34-c7d8753407ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:57.637871 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ac481-8a28-42eb-9c34-c7d8753407ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:57.638736 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ac481-8a28-42eb-9c34-c7d8753407ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:57.640371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ac481-8a28-42eb-9c34-c7d8753407ce admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:57.656225 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1195/2389] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:10:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:10:57.660452 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4594d-11e6-486a-aa46-0945ba71267c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:57.670621 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c4594d-11e6-486a-aa46-0945ba71267c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:57.675394 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533824fe-3aa1-4e18-8769-f9676fdae842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:57.676064 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533824fe-3aa1-4e18-8769-f9676fdae842 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:57.676412 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533824fe-3aa1-4e18-8769-f9676fdae842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:57.676723 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533824fe-3aa1-4e18-8769-f9676fdae842 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:57.677492 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533824fe-3aa1-4e18-8769-f9676fdae842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:57.679124 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533824fe-3aa1-4e18-8769-f9676fdae842 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:57.692499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1195/2390] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:10:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:10:57.693647 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ac481-8a28-42eb-9c34-c7d8753407ce admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:57.705160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ac481-8a28-42eb-9c34-c7d8753407ce admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:57.709760 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9d82c-b5ae-4703-a996-acaf74d04936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:57.710627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9d82c-b5ae-4703-a996-acaf74d04936 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:57.710907 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9d82c-b5ae-4703-a996-acaf74d04936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:57.711177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9d82c-b5ae-4703-a996-acaf74d04936 admin admin] PATH_INFO: `/v3/projects/687ce04ba36a4333953f86c95f920bc0/users/c62edd5a559d4ed29da25cc85e3ac1a8/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:57.727236 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9d82c-b5ae-4703-a996-acaf74d04936 admin admin] RBAC: Authorizing `identity:create_grant(project_id=687ce04ba36a4333953f86c95f920bc0, user_id=c62edd5a559d4ed29da25cc85e3ac1a8, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:57.731496 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9d82c-b5ae-4703-a996-acaf74d04936 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:57.800638 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1196/2391] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:10:57 2022] PUT /identity/v3/projects/687ce04ba36a4333953f86c95f920bc0/users/c62edd5a559d4ed29da25cc85e3ac1a8/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:10:57.811692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d2219-dbee-4855-8024-7614a03a28fe None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:57.812015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d2219-dbee-4855-8024-7614a03a28fe None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:57.812318 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d2219-dbee-4855-8024-7614a03a28fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:57.841126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-371d2219-dbee-4855-8024-7614a03a28fe None None] MFA Rules not processed for user `c62edd5a559d4ed29da25cc85e3ac1a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:10:57.842262 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-371d2219-dbee-4855-8024-7614a03a28fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:57.863021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:10:57.910772 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1196/2392] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:10:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:10:57.922162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9d82c-b5ae-4703-a996-acaf74d04936 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:57.924020 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ccc9d82c-b5ae-4703-a996-acaf74d04936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:57.987349 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9d82c-b5ae-4703-a996-acaf74d04936 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:57.991279 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293521f-fe0f-419d-99f0-b3b82c078875 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:57.991503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293521f-fe0f-419d-99f0-b3b82c078875 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:57.991782 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293521f-fe0f-419d-99f0-b3b82c078875 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:57.997353 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293521f-fe0f-419d-99f0-b3b82c078875 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:58.002497 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e293521f-fe0f-419d-99f0-b3b82c078875 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:58.004058 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e293521f-fe0f-419d-99f0-b3b82c078875 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:58.014772 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1197/2393] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:10:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:10:58.039406 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d2219-dbee-4855-8024-7614a03a28fe None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:58.104078 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d2219-dbee-4855-8024-7614a03a28fe None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:58.109158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486153b8-2834-4302-bee3-306288096752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:58.110038 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486153b8-2834-4302-bee3-306288096752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:58.110314 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486153b8-2834-4302-bee3-306288096752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:58.110586 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486153b8-2834-4302-bee3-306288096752 admin admin] PATH_INFO: `/v3/users/c62edd5a559d4ed29da25cc85e3ac1a8` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:58.111793 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486153b8-2834-4302-bee3-306288096752 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c62edd5a559d4ed29da25cc85e3ac1a8)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:58.113736 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486153b8-2834-4302-bee3-306288096752 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:58.324351 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-486153b8-2834-4302-bee3-306288096752 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c62edd5a559d4ed29da25cc85e3ac1a8'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:58.325015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-486153b8-2834-4302-bee3-306288096752 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c62edd5a559d4ed29da25cc85e3ac1a8'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:58.337964 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-486153b8-2834-4302-bee3-306288096752 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c62edd5a559d4ed29da25cc85e3ac1a8'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:58.345052 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-486153b8-2834-4302-bee3-306288096752 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c62edd5a559d4ed29da25cc85e3ac1a8'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:58.406114 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-486153b8-2834-4302-bee3-306288096752 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c62edd5a559d4ed29da25cc85e3ac1a8'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:58.412622 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1197/2394] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:10:58 2022] DELETE /identity/v3/users/c62edd5a559d4ed29da25cc85e3ac1a8 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:10:58.417663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1b8ca-0387-4646-814a-e6a8dcf71b85 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:58.424501 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1b8ca-0387-4646-814a-e6a8dcf71b85 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:58.424501 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1b8ca-0387-4646-814a-e6a8dcf71b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:58.435424 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-5cc1b8ca-0387-4646-814a-e6a8dcf71b85 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:10:58.436742 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5cc1b8ca-0387-4646-814a-e6a8dcf71b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:58.498920 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1198/2395] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:10:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:10:58.508407 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486153b8-2834-4302-bee3-306288096752 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:58.509891 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-486153b8-2834-4302-bee3-306288096752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:58.559493 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:10:58.571327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486153b8-2834-4302-bee3-306288096752 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:58.578988 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b20d9-8ed5-4bca-aad8-6a69d92cd3ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:58.578988 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b20d9-8ed5-4bca-aad8-6a69d92cd3ca service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:58.578988 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b20d9-8ed5-4bca-aad8-6a69d92cd3ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:58.578988 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b20d9-8ed5-4bca-aad8-6a69d92cd3ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:58.586233 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b20d9-8ed5-4bca-aad8-6a69d92cd3ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:58.587687 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b20d9-8ed5-4bca-aad8-6a69d92cd3ca service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:58.598265 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1198/2396] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:10:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:10:59.075069 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1b8ca-0387-4646-814a-e6a8dcf71b85 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:59.076998 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5cc1b8ca-0387-4646-814a-e6a8dcf71b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:59.094014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1b8ca-0387-4646-814a-e6a8dcf71b85 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:59.100482 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939c556-27b5-477d-9078-45e8e4a80ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:59.102000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939c556-27b5-477d-9078-45e8e4a80ae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:59.102000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939c556-27b5-477d-9078-45e8e4a80ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:59.102000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939c556-27b5-477d-9078-45e8e4a80ae4 admin admin] PATH_INFO: `/v3/projects/687ce04ba36a4333953f86c95f920bc0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:59.103466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939c556-27b5-477d-9078-45e8e4a80ae4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=687ce04ba36a4333953f86c95f920bc0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:59.106315 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939c556-27b5-477d-9078-45e8e4a80ae4 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:59.215433 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a939c556-27b5-477d-9078-45e8e4a80ae4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '687ce04ba36a4333953f86c95f920bc0', 'request_id': 'req-a939c556-27b5-477d-9078-45e8e4a80ae4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:59.290694 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a939c556-27b5-477d-9078-45e8e4a80ae4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '687ce04ba36a4333953f86c95f920bc0', 'request_id': 'req-a939c556-27b5-477d-9078-45e8e4a80ae4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:59.298127 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:10:59.299570 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a939c556-27b5-477d-9078-45e8e4a80ae4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '687ce04ba36a4333953f86c95f920bc0', 'request_id': 'req-a939c556-27b5-477d-9078-45e8e4a80ae4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:59.312179 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a939c556-27b5-477d-9078-45e8e4a80ae4 admin admin] The token cache is being invalidate because project 687ce04ba36a4333953f86c95f920bc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:10:59.312565 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a939c556-27b5-477d-9078-45e8e4a80ae4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:59.313307 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a939c556-27b5-477d-9078-45e8e4a80ae4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:10:59.314915 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1199/2397] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:10:59 2022] DELETE /identity/v3/projects/687ce04ba36a4333953f86c95f920bc0 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:10:59.319848 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc36395-69b7-482d-b222-c204022abc2f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:59.320816 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc36395-69b7-482d-b222-c204022abc2f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:59.320816 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc36395-69b7-482d-b222-c204022abc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:59.340680 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-dfc36395-69b7-482d-b222-c204022abc2f None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:10:59.341901 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-dfc36395-69b7-482d-b222-c204022abc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:59.405350 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1199/2398] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:10:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:10:59.413149 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54532fad-85ce-4058-8487-195861ab268c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:59.413442 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54532fad-85ce-4058-8487-195861ab268c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:59.413722 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54532fad-85ce-4058-8487-195861ab268c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:59.435314 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-54532fad-85ce-4058-8487-195861ab268c None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:10:59.437047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-54532fad-85ce-4058-8487-195861ab268c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:10:59.445877 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1200/2399] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:10:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:10:59.450280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc36395-69b7-482d-b222-c204022abc2f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:59.460837 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc36395-69b7-482d-b222-c204022abc2f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:59.466183 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c0c74-bf4e-4743-bf86-c3212980315a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:59.466854 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c0c74-bf4e-4743-bf86-c3212980315a admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:59.467132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c0c74-bf4e-4743-bf86-c3212980315a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:59.467406 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c0c74-bf4e-4743-bf86-c3212980315a admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:59.468136 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42c0c74-bf4e-4743-bf86-c3212980315a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:10:59.468478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42c0c74-bf4e-4743-bf86-c3212980315a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:59.470237 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42c0c74-bf4e-4743-bf86-c3212980315a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:59.483985 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1200/2400] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 20:10:59.488461 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54532fad-85ce-4058-8487-195861ab268c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:59.498801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54532fad-85ce-4058-8487-195861ab268c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:59.503765 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4c949-35d5-4bd2-9ab0-48d3b00113ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:59.504480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4c949-35d5-4bd2-9ab0-48d3b00113ff admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:59.504887 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4c949-35d5-4bd2-9ab0-48d3b00113ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:59.505033 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a4c949-35d5-4bd2-9ab0-48d3b00113ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:59.505795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4c949-35d5-4bd2-9ab0-48d3b00113ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:59.507756 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a4c949-35d5-4bd2-9ab0-48d3b00113ff admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:59.585439 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1201/2401] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:10:59 2022] POST /identity/v3/projects => generated 391 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:10:59.589903 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c0c74-bf4e-4743-bf86-c3212980315a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:59.604021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c0c74-bf4e-4743-bf86-c3212980315a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:59.664855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e729e1a4-0cfc-4da0-be79-6a091f74cb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:59.665530 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729e1a4-0cfc-4da0-be79-6a091f74cb8e admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:59.665813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729e1a4-0cfc-4da0-be79-6a091f74cb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:59.666090 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729e1a4-0cfc-4da0-be79-6a091f74cb8e admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:59.666879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e729e1a4-0cfc-4da0-be79-6a091f74cb8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:59.668838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e729e1a4-0cfc-4da0-be79-6a091f74cb8e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:59.759885 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1201/2402] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:10:59 2022] POST /identity/v3/users => generated 296 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:10:59.764869 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4c949-35d5-4bd2-9ab0-48d3b00113ff admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:59.776713 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a4c949-35d5-4bd2-9ab0-48d3b00113ff admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:59.781513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83378e-3e10-4bf0-a883-dfb0d917bc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:59.782799 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83378e-3e10-4bf0-a883-dfb0d917bc4e admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:59.783086 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83378e-3e10-4bf0-a883-dfb0d917bc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:59.783365 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83378e-3e10-4bf0-a883-dfb0d917bc4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:59.784091 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83378e-3e10-4bf0-a883-dfb0d917bc4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:59.785758 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83378e-3e10-4bf0-a883-dfb0d917bc4e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:59.796307 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1202/2403] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:10:59.800546 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e729e1a4-0cfc-4da0-be79-6a091f74cb8e admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:59.811400 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e729e1a4-0cfc-4da0-be79-6a091f74cb8e admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:59.816194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06025407-a021-49b4-b3d5-c6ee076b359a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:59.816966 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06025407-a021-49b4-b3d5-c6ee076b359a admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:59.817252 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06025407-a021-49b4-b3d5-c6ee076b359a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:59.817524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06025407-a021-49b4-b3d5-c6ee076b359a admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:59.818249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06025407-a021-49b4-b3d5-c6ee076b359a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:59.819832 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06025407-a021-49b4-b3d5-c6ee076b359a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:59.830140 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1202/2404] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:10:59.834077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83378e-3e10-4bf0-a883-dfb0d917bc4e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:10:59.844044 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83378e-3e10-4bf0-a883-dfb0d917bc4e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:10:59.848625 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7dbc3e-bca4-4281-b358-4213876b110d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:10:59.849483 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7dbc3e-bca4-4281-b358-4213876b110d admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:59.849768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7dbc3e-bca4-4281-b358-4213876b110d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:59.850040 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7dbc3e-bca4-4281-b358-4213876b110d admin admin] PATH_INFO: `/v3/projects/f4f019e27dfb4d49b4eada0d757dcdf0/users/c4f0b9ab7bfd479e827b6147b3e30070/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:59.864892 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7dbc3e-bca4-4281-b358-4213876b110d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4f019e27dfb4d49b4eada0d757dcdf0, user_id=c4f0b9ab7bfd479e827b6147b3e30070, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:10:59.867544 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7dbc3e-bca4-4281-b358-4213876b110d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:10:59.953848 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1203/2405] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:10:59 2022] PUT /identity/v3/projects/f4f019e27dfb4d49b4eada0d757dcdf0/users/c4f0b9ab7bfd479e827b6147b3e30070/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:10:59.968915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac10c2-b18e-41eb-8063-f826631aa34d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:10:59.969201 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac10c2-b18e-41eb-8063-f826631aa34d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:10:59.969475 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac10c2-b18e-41eb-8063-f826631aa34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:10:59.996997 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-baac10c2-b18e-41eb-8063-f826631aa34d None None] MFA Rules not processed for user `c4f0b9ab7bfd479e827b6147b3e30070`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:10:59.998206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-baac10c2-b18e-41eb-8063-f826631aa34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:11:00.037347 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:11:00.061950 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1203/2406] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:10:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:11:00.072762 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7dbc3e-bca4-4281-b358-4213876b110d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:11:00.074501 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1d7dbc3e-bca4-4281-b358-4213876b110d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:11:00.129929 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7dbc3e-bca4-4281-b358-4213876b110d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:11:00.133477 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8daa39-14c6-4022-999f-85e0ea8a5411 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:11:00.134101 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8daa39-14c6-4022-999f-85e0ea8a5411 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:11:00.134380 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8daa39-14c6-4022-999f-85e0ea8a5411 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:11:00.134650 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8daa39-14c6-4022-999f-85e0ea8a5411 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:11:00.142905 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8daa39-14c6-4022-999f-85e0ea8a5411 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:11:00.144404 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8daa39-14c6-4022-999f-85e0ea8a5411 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:11:00.153693 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1204/2407] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:11:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:11:10.106490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=9 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:11:10.149230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=57,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:11:20.954724 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac10c2-b18e-41eb-8063-f826631aa34d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:11:21.023728 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac10c2-b18e-41eb-8063-f826631aa34d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:11:21.028867 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40dcda-9fca-4ff9-a393-f6c6d3068921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:11:21.029989 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40dcda-9fca-4ff9-a393-f6c6d3068921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:11:21.029989 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40dcda-9fca-4ff9-a393-f6c6d3068921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:11:21.030267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40dcda-9fca-4ff9-a393-f6c6d3068921 admin admin] PATH_INFO: `/v3/users/c4f0b9ab7bfd479e827b6147b3e30070` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:11:21.031565 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f40dcda-9fca-4ff9-a393-f6c6d3068921 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4f0b9ab7bfd479e827b6147b3e30070)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:11:21.033538 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f40dcda-9fca-4ff9-a393-f6c6d3068921 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:11:21.288449 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-1f40dcda-9fca-4ff9-a393-f6c6d3068921 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4f0b9ab7bfd479e827b6147b3e30070'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:11:21.289152 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-1f40dcda-9fca-4ff9-a393-f6c6d3068921 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4f0b9ab7bfd479e827b6147b3e30070'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:11:21.301628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-1f40dcda-9fca-4ff9-a393-f6c6d3068921 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4f0b9ab7bfd479e827b6147b3e30070'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:11:21.308400 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-1f40dcda-9fca-4ff9-a393-f6c6d3068921 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4f0b9ab7bfd479e827b6147b3e30070'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:11:21.394161 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-1f40dcda-9fca-4ff9-a393-f6c6d3068921 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4f0b9ab7bfd479e827b6147b3e30070'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:11:21.400186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1204/2408] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:11:20 2022] DELETE /identity/v3/users/c4f0b9ab7bfd479e827b6147b3e30070 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:11:21.405340 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf17426-2297-48d5-8b23-d5d805368481 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:11:21.405803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf17426-2297-48d5-8b23-d5d805368481 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:11:21.405915 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf17426-2297-48d5-8b23-d5d805368481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:11:21.422482 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8cf17426-2297-48d5-8b23-d5d805368481 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:11:21.423706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8cf17426-2297-48d5-8b23-d5d805368481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:11:21.480712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1205/2409] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:11:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:11:21.490427 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40dcda-9fca-4ff9-a393-f6c6d3068921 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:11:21.491848 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1f40dcda-9fca-4ff9-a393-f6c6d3068921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:11:21.549769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40dcda-9fca-4ff9-a393-f6c6d3068921 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:11:21.553266 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a27248-b4dd-4500-9ad9-78b895021757 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:11:21.553895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a27248-b4dd-4500-9ad9-78b895021757 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:11:21.554175 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a27248-b4dd-4500-9ad9-78b895021757 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:11:21.554446 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a27248-b4dd-4500-9ad9-78b895021757 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:11:21.563795 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a27248-b4dd-4500-9ad9-78b895021757 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:11:21.565248 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a27248-b4dd-4500-9ad9-78b895021757 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:11:21.574740 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1205/2410] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:11:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:11:21.985077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf17426-2297-48d5-8b23-d5d805368481 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:11:21.986486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8cf17426-2297-48d5-8b23-d5d805368481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:11:21.999240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf17426-2297-48d5-8b23-d5d805368481 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:11:22.005015 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a1e9cd-8062-4e74-862f-63b5ada6cf42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:11:22.006287 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1e9cd-8062-4e74-862f-63b5ada6cf42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:11:22.006630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1e9cd-8062-4e74-862f-63b5ada6cf42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:11:22.006967 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a1e9cd-8062-4e74-862f-63b5ada6cf42 admin admin] PATH_INFO: `/v3/projects/f4f019e27dfb4d49b4eada0d757dcdf0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:11:22.008525 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a1e9cd-8062-4e74-862f-63b5ada6cf42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4f019e27dfb4d49b4eada0d757dcdf0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:11:22.011494 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a1e9cd-8062-4e74-862f-63b5ada6cf42 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:11:22.149211 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e8a1e9cd-8062-4e74-862f-63b5ada6cf42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4f019e27dfb4d49b4eada0d757dcdf0', 'request_id': 'req-e8a1e9cd-8062-4e74-862f-63b5ada6cf42'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:11:22.209850 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e8a1e9cd-8062-4e74-862f-63b5ada6cf42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4f019e27dfb4d49b4eada0d757dcdf0', 'request_id': 'req-e8a1e9cd-8062-4e74-862f-63b5ada6cf42'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:11:22.218337 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e8a1e9cd-8062-4e74-862f-63b5ada6cf42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4f019e27dfb4d49b4eada0d757dcdf0', 'request_id': 'req-e8a1e9cd-8062-4e74-862f-63b5ada6cf42'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:11:22.231374 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e8a1e9cd-8062-4e74-862f-63b5ada6cf42 admin admin] The token cache is being invalidate because project f4f019e27dfb4d49b4eada0d757dcdf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:11:22.232109 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e8a1e9cd-8062-4e74-862f-63b5ada6cf42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:11:22.233479 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e8a1e9cd-8062-4e74-862f-63b5ada6cf42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:11:22.235041 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1206/2411] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:11:21 2022] DELETE /identity/v3/projects/f4f019e27dfb4d49b4eada0d757dcdf0 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:11:30.845498 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d670a-ba24-4d35-b7c4-2dca0c629b16 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:11:30.846032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d670a-ba24-4d35-b7c4-2dca0c629b16 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:11:30.846032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d670a-ba24-4d35-b7c4-2dca0c629b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:11:30.863780 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e98d670a-ba24-4d35-b7c4-2dca0c629b16 None None] MFA Rules not processed for user `eafea4329d6243459dd3a6add1e91a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:11:30.865014 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e98d670a-ba24-4d35-b7c4-2dca0c629b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:11:30.882560 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:11:30.920735 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1206/2412] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:11:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:11:32.227903 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:11:32.658367 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c46eda-f82d-47a8-99a3-5f6fd43e3dbe None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:11:32.658672 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c46eda-f82d-47a8-99a3-5f6fd43e3dbe None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:11:32.658942 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c46eda-f82d-47a8-99a3-5f6fd43e3dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:11:32.677069 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-21c46eda-f82d-47a8-99a3-5f6fd43e3dbe None None] MFA Rules not processed for user `a3be90df779b4ddb8639d825fcb41234`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:11:32.678187 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-21c46eda-f82d-47a8-99a3-5f6fd43e3dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:11:32.731572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1207/2413] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:11:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:11:41.177376 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=15 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:11:42.725040 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=25 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:12:05.021956 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d670a-ba24-4d35-b7c4-2dca0c629b16 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:12:05.023406 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e98d670a-ba24-4d35-b7c4-2dca0c629b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:12:05.079875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d670a-ba24-4d35-b7c4-2dca0c629b16 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:12:05.083505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485eea10-735b-42f3-b149-35a7c8c52dea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:12:05.084195 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485eea10-735b-42f3-b149-35a7c8c52dea service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:12:05.084531 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485eea10-735b-42f3-b149-35a7c8c52dea service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:12:05.084809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485eea10-735b-42f3-b149-35a7c8c52dea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:12:05.086697 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-485eea10-735b-42f3-b149-35a7c8c52dea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:12:05.149091 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485eea10-735b-42f3-b149-35a7c8c52dea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:12:05.150531 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485eea10-735b-42f3-b149-35a7c8c52dea service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:12:05.160472 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1207/2414] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:12:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:12:09.070230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c46eda-f82d-47a8-99a3-5f6fd43e3dbe None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:12:09.071991 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-21c46eda-f82d-47a8-99a3-5f6fd43e3dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:12:09.088892 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c46eda-f82d-47a8-99a3-5f6fd43e3dbe None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:12:09.093610 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13198f3-51c4-40a2-b5a9-dfa4b5a4102e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:12:09.094481 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13198f3-51c4-40a2-b5a9-dfa4b5a4102e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:12:09.094769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13198f3-51c4-40a2-b5a9-dfa4b5a4102e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:12:09.095059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13198f3-51c4-40a2-b5a9-dfa4b5a4102e admin admin] PATH_INFO: `/v3/users/f041449191c94a5ca5a38c29dd146a15` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:12:09.096259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13198f3-51c4-40a2-b5a9-dfa4b5a4102e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f041449191c94a5ca5a38c29dd146a15)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:12:09.098218 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13198f3-51c4-40a2-b5a9-dfa4b5a4102e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:12:09.262253 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f13198f3-51c4-40a2-b5a9-dfa4b5a4102e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f041449191c94a5ca5a38c29dd146a15'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:12:09.263000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f13198f3-51c4-40a2-b5a9-dfa4b5a4102e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f041449191c94a5ca5a38c29dd146a15'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:12:09.370079 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f13198f3-51c4-40a2-b5a9-dfa4b5a4102e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f041449191c94a5ca5a38c29dd146a15'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:12:09.371864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f13198f3-51c4-40a2-b5a9-dfa4b5a4102e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f041449191c94a5ca5a38c29dd146a15'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:12:09.388677 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f13198f3-51c4-40a2-b5a9-dfa4b5a4102e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f041449191c94a5ca5a38c29dd146a15'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:12:09.406604 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1208/2415] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:12:09 2022] DELETE /identity/v3/users/f041449191c94a5ca5a38c29dd146a15 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:12:09.665296 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485eea10-735b-42f3-b149-35a7c8c52dea service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:12:09.685458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485eea10-735b-42f3-b149-35a7c8c52dea service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:12:09.759290 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb1eed6-f390-45be-b9b2-38fad66145cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:12:09.760242 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1eed6-f390-45be-b9b2-38fad66145cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:12:09.760687 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1eed6-f390-45be-b9b2-38fad66145cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:12:09.761033 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1eed6-f390-45be-b9b2-38fad66145cc admin admin] PATH_INFO: `/v3/projects/bf37c9d0240b48078038d70f678a93eb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:12:09.762293 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb1eed6-f390-45be-b9b2-38fad66145cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf37c9d0240b48078038d70f678a93eb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:12:09.764425 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb1eed6-f390-45be-b9b2-38fad66145cc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:12:09.904442 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3eb1eed6-f390-45be-b9b2-38fad66145cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf37c9d0240b48078038d70f678a93eb', 'request_id': 'req-3eb1eed6-f390-45be-b9b2-38fad66145cc'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:12:09.907998 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:12:09.910879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3eb1eed6-f390-45be-b9b2-38fad66145cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf37c9d0240b48078038d70f678a93eb', 'request_id': 'req-3eb1eed6-f390-45be-b9b2-38fad66145cc'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:12:09.916637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3eb1eed6-f390-45be-b9b2-38fad66145cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf37c9d0240b48078038d70f678a93eb', 'request_id': 'req-3eb1eed6-f390-45be-b9b2-38fad66145cc'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:12:10.001096 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3eb1eed6-f390-45be-b9b2-38fad66145cc admin admin] The token cache is being invalidate because project bf37c9d0240b48078038d70f678a93eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:12:10.001607 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3eb1eed6-f390-45be-b9b2-38fad66145cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:12:10.002274 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3eb1eed6-f390-45be-b9b2-38fad66145cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:12:10.003827 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1208/2416] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:12:09 2022] DELETE /identity/v3/projects/bf37c9d0240b48078038d70f678a93eb => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:12:10.008849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a4e17-863b-4b38-af57-80a2917f39a4 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:12:10.009420 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a4e17-863b-4b38-af57-80a2917f39a4 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:12:10.009705 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a4e17-863b-4b38-af57-80a2917f39a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:12:10.028246 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-5c3a4e17-863b-4b38-af57-80a2917f39a4 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:12:10.029491 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5c3a4e17-863b-4b38-af57-80a2917f39a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:12:10.099878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1209/2417] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:12:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:12:10.108157 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a87c51-ea28-4fb1-9f0b-516a3b59c548 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:12:10.110258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a87c51-ea28-4fb1-9f0b-516a3b59c548 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:12:10.110258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a87c51-ea28-4fb1-9f0b-516a3b59c548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:12:10.133671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-34a87c51-ea28-4fb1-9f0b-516a3b59c548 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:12:10.135128 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-34a87c51-ea28-4fb1-9f0b-516a3b59c548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:12:10.144122 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1209/2418] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:12:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:12:10.148553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a4e17-863b-4b38-af57-80a2917f39a4 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:12:10.164268 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a4e17-863b-4b38-af57-80a2917f39a4 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:12:10.170794 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3ce04-e4f5-431a-8fbe-4ee79d724fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:12:10.170995 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3ce04-e4f5-431a-8fbe-4ee79d724fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:12:10.171694 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3ce04-e4f5-431a-8fbe-4ee79d724fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:12:10.171877 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3ce04-e4f5-431a-8fbe-4ee79d724fc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:12:10.172666 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3ce04-e4f5-431a-8fbe-4ee79d724fc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:12:10.172987 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3ce04-e4f5-431a-8fbe-4ee79d724fc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:12:10.174656 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3ce04-e4f5-431a-8fbe-4ee79d724fc3 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:12:10.190169 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1210/2419] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:12:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 20:12:10.198716 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a87c51-ea28-4fb1-9f0b-516a3b59c548 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:12:10.208455 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a87c51-ea28-4fb1-9f0b-516a3b59c548 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:12:10.213086 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e8274-2199-460d-b49e-681ea9e25bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:12:10.213791 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e8274-2199-460d-b49e-681ea9e25bae admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:12:10.214190 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e8274-2199-460d-b49e-681ea9e25bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:12:10.214339 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e8274-2199-460d-b49e-681ea9e25bae admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:12:10.215038 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e8274-2199-460d-b49e-681ea9e25bae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:12:10.217920 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e8274-2199-460d-b49e-681ea9e25bae admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:12:10.308890 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1210/2420] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:12:10 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:12:10.313658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3ce04-e4f5-431a-8fbe-4ee79d724fc3 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:12:10.323652 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3ce04-e4f5-431a-8fbe-4ee79d724fc3 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:12:10.382146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba00120-0c36-4a8c-b64f-31082c54ba22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:12:10.382818 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba00120-0c36-4a8c-b64f-31082c54ba22 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:12:10.383095 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba00120-0c36-4a8c-b64f-31082c54ba22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:12:10.383369 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba00120-0c36-4a8c-b64f-31082c54ba22 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:12:10.384068 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba00120-0c36-4a8c-b64f-31082c54ba22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:12:10.386136 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba00120-0c36-4a8c-b64f-31082c54ba22 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:12:10.394901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:12:10.473207 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1211/2421] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:12:10 2022] POST /identity/v3/users => generated 288 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:12:10.477243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e8274-2199-460d-b49e-681ea9e25bae admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:12:10.487553 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e8274-2199-460d-b49e-681ea9e25bae admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:12:10.492153 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9997723-1dda-49df-a0fa-d326e2b58c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:12:10.492834 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9997723-1dda-49df-a0fa-d326e2b58c70 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:12:10.493114 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9997723-1dda-49df-a0fa-d326e2b58c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:12:10.493389 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9997723-1dda-49df-a0fa-d326e2b58c70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:12:10.494109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9997723-1dda-49df-a0fa-d326e2b58c70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:12:10.495715 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9997723-1dda-49df-a0fa-d326e2b58c70 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:12:10.506982 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1211/2422] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:12:10.510819 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba00120-0c36-4a8c-b64f-31082c54ba22 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:12:10.521677 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba00120-0c36-4a8c-b64f-31082c54ba22 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:12:10.526276 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d3ec7f-a9a6-4ac4-bf6e-fc921186466c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:12:10.526914 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3ec7f-a9a6-4ac4-bf6e-fc921186466c admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:12:10.527194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3ec7f-a9a6-4ac4-bf6e-fc921186466c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:12:10.527463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3ec7f-a9a6-4ac4-bf6e-fc921186466c admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:12:10.528176 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d3ec7f-a9a6-4ac4-bf6e-fc921186466c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:12:10.529877 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d3ec7f-a9a6-4ac4-bf6e-fc921186466c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:12:10.540845 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1212/2423] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:12:10.544895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9997723-1dda-49df-a0fa-d326e2b58c70 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:12:10.554787 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9997723-1dda-49df-a0fa-d326e2b58c70 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:12:10.559555 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f21b05-6a09-4bb8-bc96-6bbfd74b4228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:12:10.560525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f21b05-6a09-4bb8-bc96-6bbfd74b4228 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:12:10.560809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f21b05-6a09-4bb8-bc96-6bbfd74b4228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:12:10.561080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f21b05-6a09-4bb8-bc96-6bbfd74b4228 admin admin] PATH_INFO: `/v3/projects/ed44d3eff7af4bd1bc339244e2bc2fae/users/2c5e6ea89435497c8b1b06f42e08c82e/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:12:10.575794 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f21b05-6a09-4bb8-bc96-6bbfd74b4228 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed44d3eff7af4bd1bc339244e2bc2fae, user_id=2c5e6ea89435497c8b1b06f42e08c82e, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:12:10.578824 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f21b05-6a09-4bb8-bc96-6bbfd74b4228 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:12:10.652211 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1212/2424] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:12:10 2022] PUT /identity/v3/projects/ed44d3eff7af4bd1bc339244e2bc2fae/users/2c5e6ea89435497c8b1b06f42e08c82e/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:12:10.657948 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2661237-74af-49f5-b64d-9bf4b22b1d1c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:12:10.658239 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2661237-74af-49f5-b64d-9bf4b22b1d1c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:12:10.658513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2661237-74af-49f5-b64d-9bf4b22b1d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:12:10.676356 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a2661237-74af-49f5-b64d-9bf4b22b1d1c None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:12:10.677561 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a2661237-74af-49f5-b64d-9bf4b22b1d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:12:10.736044 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1213/2425] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:12:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:12:10.745344 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f21b05-6a09-4bb8-bc96-6bbfd74b4228 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:12:10.747067 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-12f21b05-6a09-4bb8-bc96-6bbfd74b4228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:12:10.807982 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f21b05-6a09-4bb8-bc96-6bbfd74b4228 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:12:10.811757 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a06bf-5a51-4cdb-885d-134217c5360c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:12:10.812442 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a06bf-5a51-4cdb-885d-134217c5360c service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:12:10.813541 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a06bf-5a51-4cdb-885d-134217c5360c service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:12:10.813541 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a06bf-5a51-4cdb-885d-134217c5360c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:12:10.822692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a06bf-5a51-4cdb-885d-134217c5360c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:12:10.824139 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a06bf-5a51-4cdb-885d-134217c5360c service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:12:10.834832 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1213/2426] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:12:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:12:14.299135 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2661237-74af-49f5-b64d-9bf4b22b1d1c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:12:14.300666 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a2661237-74af-49f5-b64d-9bf4b22b1d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:12:14.313597 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2661237-74af-49f5-b64d-9bf4b22b1d1c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:12:14.317267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d77377-bc2e-4824-9bf3-c46552c51824 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:12:14.317931 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d77377-bc2e-4824-9bf3-c46552c51824 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:12:14.318215 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d77377-bc2e-4824-9bf3-c46552c51824 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:12:14.318489 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d77377-bc2e-4824-9bf3-c46552c51824 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:12:14.320514 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a2d77377-bc2e-4824-9bf3-c46552c51824 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:12:14.395897 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d77377-bc2e-4824-9bf3-c46552c51824 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:12:14.397503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d77377-bc2e-4824-9bf3-c46552c51824 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:12:14.407475 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1214/2427] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:12:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:12:16.367789 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf441eac-92c6-42d4-9c73-c6d4a2ec396c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:12:16.368239 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf441eac-92c6-42d4-9c73-c6d4a2ec396c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:12:16.368445 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf441eac-92c6-42d4-9c73-c6d4a2ec396c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:12:16.391413 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-bf441eac-92c6-42d4-9c73-c6d4a2ec396c None None] MFA Rules not processed for user `8a5b19c8c0d44e1f93bb1dea72884cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:12:16.393041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-bf441eac-92c6-42d4-9c73-c6d4a2ec396c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:12:16.405496 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1214/2428] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:12:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:12:17.109664 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc891c7-1acf-45e2-8461-106fca8c590c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:12:17.115536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc891c7-1acf-45e2-8461-106fca8c590c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:12:17.115536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc891c7-1acf-45e2-8461-106fca8c590c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:12:17.144436 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-5dc891c7-1acf-45e2-8461-106fca8c590c None None] MFA Rules not processed for user `2c5e6ea89435497c8b1b06f42e08c82e`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:12:17.145038 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5dc891c7-1acf-45e2-8461-106fca8c590c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:12:17.221515 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1215/2429] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:12:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:12:17.244723 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf441eac-92c6-42d4-9c73-c6d4a2ec396c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:12:17.246193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-bf441eac-92c6-42d4-9c73-c6d4a2ec396c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:12:17.343866 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf441eac-92c6-42d4-9c73-c6d4a2ec396c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:12:17.352382 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e17469-d6da-44f6-9f7a-02a866793ec3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:12:17.353389 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e17469-d6da-44f6-9f7a-02a866793ec3 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:12:17.353853 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e17469-d6da-44f6-9f7a-02a866793ec3 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:12:17.354319 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e17469-d6da-44f6-9f7a-02a866793ec3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:12:17.367761 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e17469-d6da-44f6-9f7a-02a866793ec3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:12:17.369857 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e17469-d6da-44f6-9f7a-02a866793ec3 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:12:17.374088 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:12:17.381494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1215/2430] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:12:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 20:12:19.551527 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc891c7-1acf-45e2-8461-106fca8c590c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:12:19.563893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc891c7-1acf-45e2-8461-106fca8c590c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:12:19.568930 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e78000-8ece-45ec-b2ef-458b70a7e5b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:12:19.569643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e78000-8ece-45ec-b2ef-458b70a7e5b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:12:19.569938 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e78000-8ece-45ec-b2ef-458b70a7e5b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:12:19.570464 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e78000-8ece-45ec-b2ef-458b70a7e5b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:12:19.572694 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a6e78000-8ece-45ec-b2ef-458b70a7e5b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:12:19.588512 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:12:19.594792 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e78000-8ece-45ec-b2ef-458b70a7e5b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:12:19.597003 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e78000-8ece-45ec-b2ef-458b70a7e5b0 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:12:19.610677 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1216/2431] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:12:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:12:19.895213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df4e77-fa9f-43f7-9e1d-42f87b5ee7d6 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:12:19.896954 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df4e77-fa9f-43f7-9e1d-42f87b5ee7d6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:12:19.897512 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df4e77-fa9f-43f7-9e1d-42f87b5ee7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:12:19.922803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-94df4e77-fa9f-43f7-9e1d-42f87b5ee7d6 None None] MFA Rules not processed for user `2c5e6ea89435497c8b1b06f42e08c82e`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:12:19.925343 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-94df4e77-fa9f-43f7-9e1d-42f87b5ee7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:12:19.940380 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1216/2432] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:12:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:12:19.972531 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e78000-8ece-45ec-b2ef-458b70a7e5b0 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:12:19.974130 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a6e78000-8ece-45ec-b2ef-458b70a7e5b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:12:20.033898 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e78000-8ece-45ec-b2ef-458b70a7e5b0 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:12:20.037444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d5cef3-e8b7-4071-8e3b-c890dd468fac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd72f952df34ad8853fe84673a41f45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:12:20.038092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5cef3-e8b7-4071-8e3b-c890dd468fac service glance] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:12:20.038368 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5cef3-e8b7-4071-8e3b-c890dd468fac service glance] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:12:20.038638 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5cef3-e8b7-4071-8e3b-c890dd468fac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:12:20.040534 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a6d5cef3-e8b7-4071-8e3b-c890dd468fac service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:12:20.055912 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d5cef3-e8b7-4071-8e3b-c890dd468fac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:12:20.057502 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d5cef3-e8b7-4071-8e3b-c890dd468fac service glance] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:12:20.072282 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1217/2433] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:12:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 20:12:20.583995 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df4e77-fa9f-43f7-9e1d-42f87b5ee7d6 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:12:20.594999 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df4e77-fa9f-43f7-9e1d-42f87b5ee7d6 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:12:20.598845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3936d5-e6a7-4b7c-a64d-227b6e476f05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:12:20.599914 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3936d5-e6a7-4b7c-a64d-227b6e476f05 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:12:20.600400 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3936d5-e6a7-4b7c-a64d-227b6e476f05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:12:20.600845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3936d5-e6a7-4b7c-a64d-227b6e476f05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:12:20.602995 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0c3936d5-e6a7-4b7c-a64d-227b6e476f05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:12:20.613675 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3936d5-e6a7-4b7c-a64d-227b6e476f05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:12:20.615240 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3936d5-e6a7-4b7c-a64d-227b6e476f05 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:12:20.623732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1217/2434] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:12:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:12:21.263735 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d5cef3-e8b7-4071-8e3b-c890dd468fac service glance] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:12:21.265283 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a6d5cef3-e8b7-4071-8e3b-c890dd468fac service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:12:21.341663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d5cef3-e8b7-4071-8e3b-c890dd468fac service glance] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:12:21.347558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2596a5-34e3-42ad-a586-4c8367ceed69 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9715d5deeccd419b97cb0bd39e17982f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:12:21.348199 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2596a5-34e3-42ad-a586-4c8367ceed69 service placement] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:12:21.348508 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2596a5-34e3-42ad-a586-4c8367ceed69 service placement] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:12:21.348781 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2596a5-34e3-42ad-a586-4c8367ceed69 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:12:21.350509 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9f2596a5-34e3-42ad-a586-4c8367ceed69 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:12:21.361844 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2596a5-34e3-42ad-a586-4c8367ceed69 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:12:21.363426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2596a5-34e3-42ad-a586-4c8367ceed69 service placement] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:12:21.373517 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1218/2435] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:12:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:12:25.563655 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3936d5-e6a7-4b7c-a64d-227b6e476f05 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:12:25.565112 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0c3936d5-e6a7-4b7c-a64d-227b6e476f05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:12:25.582284 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3936d5-e6a7-4b7c-a64d-227b6e476f05 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:12:25.587909 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9ee5b-588a-48b7-ba36-c8230bf61012 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:12:25.588573 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9ee5b-588a-48b7-ba36-c8230bf61012 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:12:25.588854 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9ee5b-588a-48b7-ba36-c8230bf61012 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:12:25.589124 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9ee5b-588a-48b7-ba36-c8230bf61012 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:12:25.590871 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-caa9ee5b-588a-48b7-ba36-c8230bf61012 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:12:25.667825 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa9ee5b-588a-48b7-ba36-c8230bf61012 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:12:25.669527 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa9ee5b-588a-48b7-ba36-c8230bf61012 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:12:25.680995 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1218/2436] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:12:25 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:12:31.369044 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=61 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:12:32.162916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2596a5-34e3-42ad-a586-4c8367ceed69 service placement] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:12:32.173267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2596a5-34e3-42ad-a586-4c8367ceed69 service placement] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:12:32.176882 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1549574-7353-44b0-8610-ef7c86fcc33f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:12:32.177523 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1549574-7353-44b0-8610-ef7c86fcc33f service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:12:32.177805 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1549574-7353-44b0-8610-ef7c86fcc33f service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:12:32.178487 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1549574-7353-44b0-8610-ef7c86fcc33f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:12:32.179906 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a1549574-7353-44b0-8610-ef7c86fcc33f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:12:32.190973 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1549574-7353-44b0-8610-ef7c86fcc33f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:12:32.194824 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1549574-7353-44b0-8610-ef7c86fcc33f service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:12:32.200871 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1219/2437] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:12:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:12:35.676682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=38 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:12:42.196988 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=6 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:12:47.445775 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c393f8d7-6bf5-4bb8-b904-de35478668f7 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:12:47.446269 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c393f8d7-6bf5-4bb8-b904-de35478668f7 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:12:47.446368 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c393f8d7-6bf5-4bb8-b904-de35478668f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:12:47.466298 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c393f8d7-6bf5-4bb8-b904-de35478668f7 None None] MFA Rules not processed for user `1d37f369749143ce813c693adc5f2ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:12:47.467542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c393f8d7-6bf5-4bb8-b904-de35478668f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:12:47.522554 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1219/2438] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jun 23 20:12:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:12:47.678518 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1549574-7353-44b0-8610-ef7c86fcc33f service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:12:47.679924 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a1549574-7353-44b0-8610-ef7c86fcc33f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:12:47.746962 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1549574-7353-44b0-8610-ef7c86fcc33f service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:12:47.751940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f8766-ba3f-46ba-8e02-f95a4c25b82a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96db03079816461bb322633f64b9fb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:12:47.752606 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f8766-ba3f-46ba-8e02-f95a4c25b82a service swift] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:12:47.752887 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f8766-ba3f-46ba-8e02-f95a4c25b82a service swift] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:12:47.753164 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89f8766-ba3f-46ba-8e02-f95a4c25b82a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:12:47.761446 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89f8766-ba3f-46ba-8e02-f95a4c25b82a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:12:47.763091 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89f8766-ba3f-46ba-8e02-f95a4c25b82a service swift] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:12:47.771946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1220/2439] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jun 23 20:12:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 20:12:57.515078 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=25 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:12:57.768732 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=32 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:13:02.425284 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c393f8d7-6bf5-4bb8-b904-de35478668f7 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:02.438482 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c393f8d7-6bf5-4bb8-b904-de35478668f7 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:02.443732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe07676a-ccc5-4eef-b73f-57660895b2e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:02.444402 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07676a-ccc5-4eef-b73f-57660895b2e7 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:02.444682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07676a-ccc5-4eef-b73f-57660895b2e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:02.444953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe07676a-ccc5-4eef-b73f-57660895b2e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:02.446688 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fe07676a-ccc5-4eef-b73f-57660895b2e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:13:02.459240 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe07676a-ccc5-4eef-b73f-57660895b2e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:02.460850 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe07676a-ccc5-4eef-b73f-57660895b2e7 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:02.471106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1220/2440] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:13:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:13:04.092673 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f8766-ba3f-46ba-8e02-f95a4c25b82a service swift] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:04.102290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89f8766-ba3f-46ba-8e02-f95a4c25b82a service swift] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:04.105955 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7acce8-ea67-4752-bcea-d5b735a3a5d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:04.106586 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7acce8-ea67-4752-bcea-d5b735a3a5d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:04.106868 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7acce8-ea67-4752-bcea-d5b735a3a5d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:04.107145 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7acce8-ea67-4752-bcea-d5b735a3a5d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:04.108962 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1a7acce8-ea67-4752-bcea-d5b735a3a5d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:13:04.119175 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7acce8-ea67-4752-bcea-d5b735a3a5d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:04.122376 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7acce8-ea67-4752-bcea-d5b735a3a5d8 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:04.133014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1221/2441] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:13:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:13:07.161982 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a61241-7430-40d6-8d5e-7d548bc5d8d4 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:07.162482 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a61241-7430-40d6-8d5e-7d548bc5d8d4 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:07.162591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a61241-7430-40d6-8d5e-7d548bc5d8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:07.180249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-44a61241-7430-40d6-8d5e-7d548bc5d8d4 None None] MFA Rules not processed for user `1d37f369749143ce813c693adc5f2ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:13:07.181488 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-44a61241-7430-40d6-8d5e-7d548bc5d8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:13:07.188805 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1221/2442] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jun 23 20:13:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:13:07.200754 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7acce8-ea67-4752-bcea-d5b735a3a5d8 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:07.213460 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7acce8-ea67-4752-bcea-d5b735a3a5d8 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:07.218354 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dba9e0-9ac3-4da5-af09-5469e367d4ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96db03079816461bb322633f64b9fb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:07.219550 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dba9e0-9ac3-4da5-af09-5469e367d4ca service swift] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:07.219834 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dba9e0-9ac3-4da5-af09-5469e367d4ca service swift] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:07.220108 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dba9e0-9ac3-4da5-af09-5469e367d4ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:07.228337 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dba9e0-9ac3-4da5-af09-5469e367d4ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:07.229939 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dba9e0-9ac3-4da5-af09-5469e367d4ca service swift] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:07.238357 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1222/2443] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jun 23 20:13:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 20:13:17.175604 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=9 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:13:17.234943 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=12 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:13:24.154878 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90caac-e518-4aaf-8c4f-1e7c272f8a5b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:24.155363 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90caac-e518-4aaf-8c4f-1e7c272f8a5b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:24.155454 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90caac-e518-4aaf-8c4f-1e7c272f8a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:24.172527 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ce90caac-e518-4aaf-8c4f-1e7c272f8a5b None None] MFA Rules not processed for user `1d37f369749143ce813c693adc5f2ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:13:24.173705 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ce90caac-e518-4aaf-8c4f-1e7c272f8a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:13:24.181213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1222/2444] 10.222.0.44 () {58 vars in 967 bytes} [Thu Jun 23 20:13:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:13:24.192891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dba9e0-9ac3-4da5-af09-5469e367d4ca service swift] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:24.203892 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dba9e0-9ac3-4da5-af09-5469e367d4ca service swift] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:24.209182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61148791-1d0f-4f25-a70d-242792021496 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96db03079816461bb322633f64b9fb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:24.209838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61148791-1d0f-4f25-a70d-242792021496 service swift] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:24.210118 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61148791-1d0f-4f25-a70d-242792021496 service swift] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:24.210389 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61148791-1d0f-4f25-a70d-242792021496 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:24.218294 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61148791-1d0f-4f25-a70d-242792021496 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:24.219875 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61148791-1d0f-4f25-a70d-242792021496 service swift] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:24.229959 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1223/2445] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jun 23 20:13:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 20:13:28.205626 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90caac-e518-4aaf-8c4f-1e7c272f8a5b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:28.207021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ce90caac-e518-4aaf-8c4f-1e7c272f8a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:13:28.263700 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90caac-e518-4aaf-8c4f-1e7c272f8a5b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:28.267213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80d0ac-b053-40e9-bce3-4a62ee614b53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:28.267848 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80d0ac-b053-40e9-bce3-4a62ee614b53 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:28.268135 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80d0ac-b053-40e9-bce3-4a62ee614b53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:28.268436 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80d0ac-b053-40e9-bce3-4a62ee614b53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:28.270180 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1d80d0ac-b053-40e9-bce3-4a62ee614b53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:13:28.328777 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80d0ac-b053-40e9-bce3-4a62ee614b53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:28.330226 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80d0ac-b053-40e9-bce3-4a62ee614b53 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:28.340111 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1223/2446] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:13:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:13:34.224701 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=6 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:13:36.733456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e885d2e-a9c7-4e03-a024-58303ec11249 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:36.734047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e885d2e-a9c7-4e03-a024-58303ec11249 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:36.734047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e885d2e-a9c7-4e03-a024-58303ec11249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:36.751475 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-0e885d2e-a9c7-4e03-a024-58303ec11249 None None] MFA Rules not processed for user `eafea4329d6243459dd3a6add1e91a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:13:36.752689 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0e885d2e-a9c7-4e03-a024-58303ec11249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:13:36.759916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1224/2447] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:13:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:13:38.282346 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80d0ac-b053-40e9-bce3-4a62ee614b53 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:38.295694 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80d0ac-b053-40e9-bce3-4a62ee614b53 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:38.301606 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec5e32-4a49-44e3-9dd5-a6702518691f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:38.302277 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec5e32-4a49-44e3-9dd5-a6702518691f service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:38.302559 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec5e32-4a49-44e3-9dd5-a6702518691f service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:38.302833 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec5e32-4a49-44e3-9dd5-a6702518691f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:38.305082 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c8ec5e32-4a49-44e3-9dd5-a6702518691f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:13:38.318291 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec5e32-4a49-44e3-9dd5-a6702518691f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:38.319815 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec5e32-4a49-44e3-9dd5-a6702518691f service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:38.334583 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1224/2448] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:13:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:13:45.881217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e885d2e-a9c7-4e03-a024-58303ec11249 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:45.891563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e885d2e-a9c7-4e03-a024-58303ec11249 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:45.895138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eed03e-29a2-4037-8c8a-f5918cb2aed6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:45.895770 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eed03e-29a2-4037-8c8a-f5918cb2aed6 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:45.896048 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eed03e-29a2-4037-8c8a-f5918cb2aed6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:45.896344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eed03e-29a2-4037-8c8a-f5918cb2aed6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:45.898087 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-59eed03e-29a2-4037-8c8a-f5918cb2aed6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:13:45.909581 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eed03e-29a2-4037-8c8a-f5918cb2aed6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:45.911007 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eed03e-29a2-4037-8c8a-f5918cb2aed6 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:45.920516 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1225/2449] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:13:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:13:48.327044 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=57 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:13:49.570011 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec5e32-4a49-44e3-9dd5-a6702518691f service nova] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:49.571495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c8ec5e32-4a49-44e3-9dd5-a6702518691f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:13:49.586043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec5e32-4a49-44e3-9dd5-a6702518691f service nova] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:49.591528 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb3d1d-1ee0-4508-867d-493c9d4ffd38 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9715d5deeccd419b97cb0bd39e17982f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:49.592162 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb3d1d-1ee0-4508-867d-493c9d4ffd38 service placement] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:49.592493 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb3d1d-1ee0-4508-867d-493c9d4ffd38 service placement] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:49.592765 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb3d1d-1ee0-4508-867d-493c9d4ffd38 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:49.594498 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-70eb3d1d-1ee0-4508-867d-493c9d4ffd38 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:13:49.606593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb3d1d-1ee0-4508-867d-493c9d4ffd38 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:49.608161 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb3d1d-1ee0-4508-867d-493c9d4ffd38 service placement] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:49.618172 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1225/2450] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:13:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:13:49.724073 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eed03e-29a2-4037-8c8a-f5918cb2aed6 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:49.725647 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-59eed03e-29a2-4037-8c8a-f5918cb2aed6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:13:49.738983 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eed03e-29a2-4037-8c8a-f5918cb2aed6 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:49.743752 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9962be-6a1c-433e-ae26-538854201cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:49.744674 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9962be-6a1c-433e-ae26-538854201cb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:49.744948 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9962be-6a1c-433e-ae26-538854201cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:49.745211 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9962be-6a1c-433e-ae26-538854201cb0 admin admin] PATH_INFO: `/v3/users/8a5b19c8c0d44e1f93bb1dea72884cc3` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:49.746413 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9962be-6a1c-433e-ae26-538854201cb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a5b19c8c0d44e1f93bb1dea72884cc3)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:49.748361 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9962be-6a1c-433e-ae26-538854201cb0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:49.971530 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2b9962be-6a1c-433e-ae26-538854201cb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a5b19c8c0d44e1f93bb1dea72884cc3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:13:49.972143 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2b9962be-6a1c-433e-ae26-538854201cb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a5b19c8c0d44e1f93bb1dea72884cc3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:13:50.041012 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2b9962be-6a1c-433e-ae26-538854201cb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a5b19c8c0d44e1f93bb1dea72884cc3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:13:50.041696 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2b9962be-6a1c-433e-ae26-538854201cb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a5b19c8c0d44e1f93bb1dea72884cc3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:13:50.054446 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2b9962be-6a1c-433e-ae26-538854201cb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a5b19c8c0d44e1f93bb1dea72884cc3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:13:50.066626 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1226/2451] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:13:49 2022] DELETE /identity/v3/users/8a5b19c8c0d44e1f93bb1dea72884cc3 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:13:50.340530 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb3d1d-1ee0-4508-867d-493c9d4ffd38 service placement] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:50.352248 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb3d1d-1ee0-4508-867d-493c9d4ffd38 service placement] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:50.452331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f23521-9736-4dc3-98ed-4e599db5dbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:50.453198 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f23521-9736-4dc3-98ed-4e599db5dbb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:50.453476 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f23521-9736-4dc3-98ed-4e599db5dbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:50.453751 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f23521-9736-4dc3-98ed-4e599db5dbb9 admin admin] PATH_INFO: `/v3/projects/bd810755b80346ae9219a2d0c61a8a06` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:50.454934 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f23521-9736-4dc3-98ed-4e599db5dbb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd810755b80346ae9219a2d0c61a8a06)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:50.457170 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f23521-9736-4dc3-98ed-4e599db5dbb9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:50.585577 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-46f23521-9736-4dc3-98ed-4e599db5dbb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd810755b80346ae9219a2d0c61a8a06', 'request_id': 'req-46f23521-9736-4dc3-98ed-4e599db5dbb9'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:13:50.591836 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-46f23521-9736-4dc3-98ed-4e599db5dbb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd810755b80346ae9219a2d0c61a8a06', 'request_id': 'req-46f23521-9736-4dc3-98ed-4e599db5dbb9'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:13:50.597768 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-46f23521-9736-4dc3-98ed-4e599db5dbb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd810755b80346ae9219a2d0c61a8a06', 'request_id': 'req-46f23521-9736-4dc3-98ed-4e599db5dbb9'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:13:50.682881 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-46f23521-9736-4dc3-98ed-4e599db5dbb9 admin admin] The token cache is being invalidate because project bd810755b80346ae9219a2d0c61a8a06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:13:50.683162 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-46f23521-9736-4dc3-98ed-4e599db5dbb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:13:50.684906 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-46f23521-9736-4dc3-98ed-4e599db5dbb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:13:50.687767 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1226/2452] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:13:50 2022] DELETE /identity/v3/projects/bd810755b80346ae9219a2d0c61a8a06 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:13:50.693908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27900d-bb9b-422f-9bc9-dde6991543d7 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:50.694293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27900d-bb9b-422f-9bc9-dde6991543d7 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:50.694569 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27900d-bb9b-422f-9bc9-dde6991543d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:50.713781 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3e27900d-bb9b-422f-9bc9-dde6991543d7 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:13:50.714999 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3e27900d-bb9b-422f-9bc9-dde6991543d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:13:50.773436 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1227/2453] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:13:50.781512 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0ebad-d826-4cc1-ba15-dbb9b724d482 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:50.781803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0ebad-d826-4cc1-ba15-dbb9b724d482 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:50.782105 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0ebad-d826-4cc1-ba15-dbb9b724d482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:50.800689 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-3dc0ebad-d826-4cc1-ba15-dbb9b724d482 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:13:50.801907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3dc0ebad-d826-4cc1-ba15-dbb9b724d482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:13:50.810366 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1227/2454] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:13:50.814521 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27900d-bb9b-422f-9bc9-dde6991543d7 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:50.823999 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27900d-bb9b-422f-9bc9-dde6991543d7 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:50.828571 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e512283-1116-4c94-8f2c-a0f1a1074095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:50.829221 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e512283-1116-4c94-8f2c-a0f1a1074095 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:50.829502 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e512283-1116-4c94-8f2c-a0f1a1074095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:50.829780 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e512283-1116-4c94-8f2c-a0f1a1074095 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:50.830508 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e512283-1116-4c94-8f2c-a0f1a1074095 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:13:50.830811 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e512283-1116-4c94-8f2c-a0f1a1074095 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:50.832714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e512283-1116-4c94-8f2c-a0f1a1074095 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:50.844524 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1228/2455] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:13:50.848605 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc0ebad-d826-4cc1-ba15-dbb9b724d482 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:50.858234 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc0ebad-d826-4cc1-ba15-dbb9b724d482 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:50.863752 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb02c9-8c78-4906-947f-9f8ff215d78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:50.864423 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb02c9-8c78-4906-947f-9f8ff215d78f admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:50.864705 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb02c9-8c78-4906-947f-9f8ff215d78f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:50.864976 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb02c9-8c78-4906-947f-9f8ff215d78f admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:50.865658 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb02c9-8c78-4906-947f-9f8ff215d78f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:50.867617 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb02c9-8c78-4906-947f-9f8ff215d78f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:50.971804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1228/2456] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:13:50 2022] POST /identity/v3/projects => generated 367 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:13:50.972879 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e512283-1116-4c94-8f2c-a0f1a1074095 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:50.985351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e512283-1116-4c94-8f2c-a0f1a1074095 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:51.024686 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:13:51.044056 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356fff31-1cac-4589-bb82-cc35192bc113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:51.049784 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356fff31-1cac-4589-bb82-cc35192bc113 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:51.049784 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356fff31-1cac-4589-bb82-cc35192bc113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:51.049784 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356fff31-1cac-4589-bb82-cc35192bc113 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:51.049784 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356fff31-1cac-4589-bb82-cc35192bc113 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:51.049784 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356fff31-1cac-4589-bb82-cc35192bc113 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:51.180926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1229/2457] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:13:50 2022] POST /identity/v3/users => generated 284 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:13:51.184904 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb02c9-8c78-4906-947f-9f8ff215d78f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:51.195203 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb02c9-8c78-4906-947f-9f8ff215d78f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:51.199777 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937846a2-cd45-4ece-bcbd-63a495f9b4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:51.200457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937846a2-cd45-4ece-bcbd-63a495f9b4b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:51.200736 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937846a2-cd45-4ece-bcbd-63a495f9b4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:51.201002 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937846a2-cd45-4ece-bcbd-63a495f9b4b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:51.201714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937846a2-cd45-4ece-bcbd-63a495f9b4b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:51.203329 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937846a2-cd45-4ece-bcbd-63a495f9b4b8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:51.213403 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1229/2458] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:13:51.217732 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356fff31-1cac-4589-bb82-cc35192bc113 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:51.227691 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356fff31-1cac-4589-bb82-cc35192bc113 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:51.233068 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd034cd-0b7c-4d84-befd-0b6ef96f2e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:51.237775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd034cd-0b7c-4d84-befd-0b6ef96f2e50 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:51.237775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd034cd-0b7c-4d84-befd-0b6ef96f2e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:51.237775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd034cd-0b7c-4d84-befd-0b6ef96f2e50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:51.237775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd034cd-0b7c-4d84-befd-0b6ef96f2e50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:51.237775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd034cd-0b7c-4d84-befd-0b6ef96f2e50 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:51.248104 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1230/2459] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:13:51.252118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937846a2-cd45-4ece-bcbd-63a495f9b4b8 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:51.262044 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937846a2-cd45-4ece-bcbd-63a495f9b4b8 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:51.266567 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1161366d-51cd-4e7d-9c7d-271a3f2cff62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:51.272194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161366d-51cd-4e7d-9c7d-271a3f2cff62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:51.272194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161366d-51cd-4e7d-9c7d-271a3f2cff62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:51.272194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161366d-51cd-4e7d-9c7d-271a3f2cff62 admin admin] PATH_INFO: `/v3/projects/0ebd476f75ca4d0c8932b3b84ac81763/users/28e652fe5cdd46e39c062d8976149af5/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:51.283600 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1161366d-51cd-4e7d-9c7d-271a3f2cff62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ebd476f75ca4d0c8932b3b84ac81763, user_id=28e652fe5cdd46e39c062d8976149af5, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:51.286341 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1161366d-51cd-4e7d-9c7d-271a3f2cff62 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:51.361243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1230/2460] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:13:51 2022] PUT /identity/v3/projects/0ebd476f75ca4d0c8932b3b84ac81763/users/28e652fe5cdd46e39c062d8976149af5/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:13:51.371289 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b514f3-661d-47d0-8fad-10443d27cb01 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:51.371589 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b514f3-661d-47d0-8fad-10443d27cb01 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:51.371868 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b514f3-661d-47d0-8fad-10443d27cb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:51.398266 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-74b514f3-661d-47d0-8fad-10443d27cb01 None None] MFA Rules not processed for user `28e652fe5cdd46e39c062d8976149af5`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:13:51.399464 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-74b514f3-661d-47d0-8fad-10443d27cb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:13:51.462537 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1231/2461] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:13:51 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:13:51.467751 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1161366d-51cd-4e7d-9c7d-271a3f2cff62 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:51.473595 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:13:51.533425 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1161366d-51cd-4e7d-9c7d-271a3f2cff62 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:51.538098 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257819b-a6ea-4c49-84f5-0e8d5b72dd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:51.538782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257819b-a6ea-4c49-84f5-0e8d5b72dd72 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:51.539061 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257819b-a6ea-4c49-84f5-0e8d5b72dd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:51.539334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257819b-a6ea-4c49-84f5-0e8d5b72dd72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:51.540033 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257819b-a6ea-4c49-84f5-0e8d5b72dd72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:51.541990 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257819b-a6ea-4c49-84f5-0e8d5b72dd72 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:51.650624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1231/2462] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:13:51 2022] POST /identity/v3/projects => generated 369 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:13:51.655216 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b514f3-661d-47d0-8fad-10443d27cb01 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:51.665710 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b514f3-661d-47d0-8fad-10443d27cb01 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:51.720380 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57748b-1259-4388-a828-d393b324911c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:51.721039 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57748b-1259-4388-a828-d393b324911c admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:51.721341 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57748b-1259-4388-a828-d393b324911c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:51.721599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57748b-1259-4388-a828-d393b324911c admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:51.722274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a57748b-1259-4388-a828-d393b324911c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:51.724204 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a57748b-1259-4388-a828-d393b324911c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:51.791021 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1232/2463] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:13:51 2022] POST /identity/v3/users => generated 291 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:13:51.795226 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257819b-a6ea-4c49-84f5-0e8d5b72dd72 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:51.805624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257819b-a6ea-4c49-84f5-0e8d5b72dd72 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:51.810717 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de82abc1-5d45-4b24-8083-273cee91a07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:51.811367 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82abc1-5d45-4b24-8083-273cee91a07b admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:51.811652 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82abc1-5d45-4b24-8083-273cee91a07b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:51.811929 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82abc1-5d45-4b24-8083-273cee91a07b admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:51.812683 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de82abc1-5d45-4b24-8083-273cee91a07b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:51.814279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de82abc1-5d45-4b24-8083-273cee91a07b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:51.825426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1232/2464] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:13:51.829482 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57748b-1259-4388-a828-d393b324911c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:51.839838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57748b-1259-4388-a828-d393b324911c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:51.844326 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de936bb1-c819-49ed-9872-32ba297f7490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:51.845214 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de936bb1-c819-49ed-9872-32ba297f7490 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:51.845490 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de936bb1-c819-49ed-9872-32ba297f7490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:51.845759 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de936bb1-c819-49ed-9872-32ba297f7490 admin admin] PATH_INFO: `/v3/projects/afb9f0f168b643cab83c9b8a393464af/users/84f9d9b276454dae8dfa2aa1986e4466/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:51.859977 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de936bb1-c819-49ed-9872-32ba297f7490 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afb9f0f168b643cab83c9b8a393464af, user_id=84f9d9b276454dae8dfa2aa1986e4466, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:51.862729 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de936bb1-c819-49ed-9872-32ba297f7490 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:51.940414 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1233/2465] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:13:51 2022] PUT /identity/v3/projects/afb9f0f168b643cab83c9b8a393464af/users/84f9d9b276454dae8dfa2aa1986e4466/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:13:51.944259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de82abc1-5d45-4b24-8083-273cee91a07b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:51.955053 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de82abc1-5d45-4b24-8083-273cee91a07b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:52.014249 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e3379-1ba1-4b09-a167-5b277a7e68de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:52.014915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e3379-1ba1-4b09-a167-5b277a7e68de admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:52.015200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e3379-1ba1-4b09-a167-5b277a7e68de admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:52.015478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e3379-1ba1-4b09-a167-5b277a7e68de admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:52.016197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e3379-1ba1-4b09-a167-5b277a7e68de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:52.017822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e3379-1ba1-4b09-a167-5b277a7e68de admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:52.027969 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1233/2466] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:13:52.031882 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de936bb1-c819-49ed-9872-32ba297f7490 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:52.042494 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de936bb1-c819-49ed-9872-32ba297f7490 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:52.047118 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f2c25-f979-45a4-b21d-04c73f64cb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:52.047990 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f2c25-f979-45a4-b21d-04c73f64cb6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:52.048269 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f2c25-f979-45a4-b21d-04c73f64cb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:52.048571 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f2c25-f979-45a4-b21d-04c73f64cb6a admin admin] PATH_INFO: `/v3/projects/afb9f0f168b643cab83c9b8a393464af/users/84f9d9b276454dae8dfa2aa1986e4466/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:52.050501 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22f2c25-f979-45a4-b21d-04c73f64cb6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=afb9f0f168b643cab83c9b8a393464af, user_id=84f9d9b276454dae8dfa2aa1986e4466, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:52.053251 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22f2c25-f979-45a4-b21d-04c73f64cb6a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:52.059904 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:13:52.064242 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1234/2467] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:13:52 2022] PUT /identity/v3/projects/afb9f0f168b643cab83c9b8a393464af/users/84f9d9b276454dae8dfa2aa1986e4466/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:13:52.076190 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9493e09-15f1-455e-a9b5-74195def0d37 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:52.082645 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9493e09-15f1-455e-a9b5-74195def0d37 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:52.082645 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9493e09-15f1-455e-a9b5-74195def0d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:52.103653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d9493e09-15f1-455e-a9b5-74195def0d37 None None] MFA Rules not processed for user `84f9d9b276454dae8dfa2aa1986e4466`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:13:52.104996 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d9493e09-15f1-455e-a9b5-74195def0d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:13:52.192768 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:13:52.196760 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1234/2468] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:13:52 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:13:52.208783 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f2c25-f979-45a4-b21d-04c73f64cb6a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:52.210489 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e22f2c25-f979-45a4-b21d-04c73f64cb6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:13:52.269971 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f2c25-f979-45a4-b21d-04c73f64cb6a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:52.273973 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c910154-93c2-4262-b214-2391452ef262 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:52.274806 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c910154-93c2-4262-b214-2391452ef262 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:52.275246 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c910154-93c2-4262-b214-2391452ef262 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:52.275686 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c910154-93c2-4262-b214-2391452ef262 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:52.290350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c910154-93c2-4262-b214-2391452ef262 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:52.290350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c910154-93c2-4262-b214-2391452ef262 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:52.300796 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1235/2469] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:13:52 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:13:52.414824 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9493e09-15f1-455e-a9b5-74195def0d37 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:52.416199 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d9493e09-15f1-455e-a9b5-74195def0d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:13:52.428253 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9493e09-15f1-455e-a9b5-74195def0d37 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:52.433679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c110e-9e22-4ef8-aad0-a41b482cfba8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:52.433679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c110e-9e22-4ef8-aad0-a41b482cfba8 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:52.433679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c110e-9e22-4ef8-aad0-a41b482cfba8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:52.433679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c110e-9e22-4ef8-aad0-a41b482cfba8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:52.501779 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080c110e-9e22-4ef8-aad0-a41b482cfba8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:52.503158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080c110e-9e22-4ef8-aad0-a41b482cfba8 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:52.512229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1235/2470] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:13:52 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:13:57.513235 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c910154-93c2-4262-b214-2391452ef262 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:57.579995 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c910154-93c2-4262-b214-2391452ef262 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:57.585100 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc8b62-7716-4c32-84c8-d6aece99b4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:57.586010 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc8b62-7716-4c32-84c8-d6aece99b4e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:57.586288 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc8b62-7716-4c32-84c8-d6aece99b4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:57.586561 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc8b62-7716-4c32-84c8-d6aece99b4e1 admin admin] PATH_INFO: `/v3/users/28e652fe5cdd46e39c062d8976149af5` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:57.587755 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc8b62-7716-4c32-84c8-d6aece99b4e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28e652fe5cdd46e39c062d8976149af5)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:57.589961 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc8b62-7716-4c32-84c8-d6aece99b4e1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:57.798234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8ddc8b62-7716-4c32-84c8-d6aece99b4e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28e652fe5cdd46e39c062d8976149af5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:13:57.799042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8ddc8b62-7716-4c32-84c8-d6aece99b4e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28e652fe5cdd46e39c062d8976149af5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:13:57.865927 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8ddc8b62-7716-4c32-84c8-d6aece99b4e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28e652fe5cdd46e39c062d8976149af5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:13:57.866654 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8ddc8b62-7716-4c32-84c8-d6aece99b4e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28e652fe5cdd46e39c062d8976149af5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:13:57.880376 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8ddc8b62-7716-4c32-84c8-d6aece99b4e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28e652fe5cdd46e39c062d8976149af5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:13:57.893582 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1236/2471] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:13:57 2022] DELETE /identity/v3/users/28e652fe5cdd46e39c062d8976149af5 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:13:57.898123 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfbb45-4828-4ecc-8aab-1529f82ec0b2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:57.898632 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfbb45-4828-4ecc-8aab-1529f82ec0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:57.898761 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfbb45-4828-4ecc-8aab-1529f82ec0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:57.916238 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-38cfbb45-4828-4ecc-8aab-1529f82ec0b2 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:13:57.917602 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-38cfbb45-4828-4ecc-8aab-1529f82ec0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:13:57.982692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1236/2472] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:13:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:13:57.992789 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc8b62-7716-4c32-84c8-d6aece99b4e1 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:57.994251 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8ddc8b62-7716-4c32-84c8-d6aece99b4e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:13:58.022703 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:13:58.055753 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc8b62-7716-4c32-84c8-d6aece99b4e1 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:58.060042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e18fa2-aa8b-4882-b15b-865997427c92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:58.060888 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e18fa2-aa8b-4882-b15b-865997427c92 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:58.061642 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e18fa2-aa8b-4882-b15b-865997427c92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:58.062048 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e18fa2-aa8b-4882-b15b-865997427c92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:58.071235 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e18fa2-aa8b-4882-b15b-865997427c92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:58.072819 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e18fa2-aa8b-4882-b15b-865997427c92 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:58.083503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1237/2473] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:13:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 20:13:58.621205 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cfbb45-4828-4ecc-8aab-1529f82ec0b2 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:58.622654 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-38cfbb45-4828-4ecc-8aab-1529f82ec0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:13:58.634880 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cfbb45-4828-4ecc-8aab-1529f82ec0b2 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:58.639485 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1943ead-0bc4-46db-8ddd-be5b3bcc78e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:58.640358 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1943ead-0bc4-46db-8ddd-be5b3bcc78e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:58.640640 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1943ead-0bc4-46db-8ddd-be5b3bcc78e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:58.640918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1943ead-0bc4-46db-8ddd-be5b3bcc78e2 admin admin] PATH_INFO: `/v3/projects/0ebd476f75ca4d0c8932b3b84ac81763` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:58.642084 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1943ead-0bc4-46db-8ddd-be5b3bcc78e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ebd476f75ca4d0c8932b3b84ac81763)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:58.643973 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1943ead-0bc4-46db-8ddd-be5b3bcc78e2 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:58.761071 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c1943ead-0bc4-46db-8ddd-be5b3bcc78e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ebd476f75ca4d0c8932b3b84ac81763', 'request_id': 'req-c1943ead-0bc4-46db-8ddd-be5b3bcc78e2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:13:58.767326 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c1943ead-0bc4-46db-8ddd-be5b3bcc78e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ebd476f75ca4d0c8932b3b84ac81763', 'request_id': 'req-c1943ead-0bc4-46db-8ddd-be5b3bcc78e2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:13:58.773158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c1943ead-0bc4-46db-8ddd-be5b3bcc78e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ebd476f75ca4d0c8932b3b84ac81763', 'request_id': 'req-c1943ead-0bc4-46db-8ddd-be5b3bcc78e2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:13:58.865033 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c1943ead-0bc4-46db-8ddd-be5b3bcc78e2 admin admin] The token cache is being invalidate because project 0ebd476f75ca4d0c8932b3b84ac81763 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:13:58.865408 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c1943ead-0bc4-46db-8ddd-be5b3bcc78e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:13:58.866069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c1943ead-0bc4-46db-8ddd-be5b3bcc78e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:13:58.867681 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1237/2474] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:13:58 2022] DELETE /identity/v3/projects/0ebd476f75ca4d0c8932b3b84ac81763 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:13:58.871976 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e18fa2-aa8b-4882-b15b-865997427c92 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:58.882668 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e18fa2-aa8b-4882-b15b-865997427c92 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:58.942843 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78628a05-1af5-4af3-b1b4-52d3bb0cefb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:58.943718 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78628a05-1af5-4af3-b1b4-52d3bb0cefb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:58.944121 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78628a05-1af5-4af3-b1b4-52d3bb0cefb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:58.944276 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78628a05-1af5-4af3-b1b4-52d3bb0cefb9 admin admin] PATH_INFO: `/v3/users/84f9d9b276454dae8dfa2aa1986e4466` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:58.945554 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78628a05-1af5-4af3-b1b4-52d3bb0cefb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84f9d9b276454dae8dfa2aa1986e4466)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:58.949301 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78628a05-1af5-4af3-b1b4-52d3bb0cefb9 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:59.128419 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-78628a05-1af5-4af3-b1b4-52d3bb0cefb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84f9d9b276454dae8dfa2aa1986e4466'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:13:59.129903 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-78628a05-1af5-4af3-b1b4-52d3bb0cefb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84f9d9b276454dae8dfa2aa1986e4466'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:13:59.191823 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-78628a05-1af5-4af3-b1b4-52d3bb0cefb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84f9d9b276454dae8dfa2aa1986e4466'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:13:59.192532 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-78628a05-1af5-4af3-b1b4-52d3bb0cefb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84f9d9b276454dae8dfa2aa1986e4466'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:13:59.208881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-78628a05-1af5-4af3-b1b4-52d3bb0cefb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84f9d9b276454dae8dfa2aa1986e4466'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:13:59.225041 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1238/2475] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:13:58 2022] DELETE /identity/v3/users/84f9d9b276454dae8dfa2aa1986e4466 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:13:59.751019 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1943ead-0bc4-46db-8ddd-be5b3bcc78e2 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:13:59.752504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c1943ead-0bc4-46db-8ddd-be5b3bcc78e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:13:59.765892 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1943ead-0bc4-46db-8ddd-be5b3bcc78e2 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:13:59.785813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:13:59.825592 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7cb6d5-9939-4019-8956-500796378189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:13:59.826576 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cb6d5-9939-4019-8956-500796378189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:13:59.826862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cb6d5-9939-4019-8956-500796378189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:13:59.827138 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cb6d5-9939-4019-8956-500796378189 admin admin] PATH_INFO: `/v3/projects/afb9f0f168b643cab83c9b8a393464af` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:13:59.828373 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7cb6d5-9939-4019-8956-500796378189 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afb9f0f168b643cab83c9b8a393464af)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:13:59.830300 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7cb6d5-9939-4019-8956-500796378189 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:13:59.977160 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-af7cb6d5-9939-4019-8956-500796378189 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afb9f0f168b643cab83c9b8a393464af', 'request_id': 'req-af7cb6d5-9939-4019-8956-500796378189'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:13:59.983960 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-af7cb6d5-9939-4019-8956-500796378189 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afb9f0f168b643cab83c9b8a393464af', 'request_id': 'req-af7cb6d5-9939-4019-8956-500796378189'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:13:59.989375 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-af7cb6d5-9939-4019-8956-500796378189 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afb9f0f168b643cab83c9b8a393464af', 'request_id': 'req-af7cb6d5-9939-4019-8956-500796378189'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:00.070940 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-af7cb6d5-9939-4019-8956-500796378189 admin admin] The token cache is being invalidate because project afb9f0f168b643cab83c9b8a393464af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:14:00.071222 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-af7cb6d5-9939-4019-8956-500796378189 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:00.071854 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-af7cb6d5-9939-4019-8956-500796378189 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:00.073476 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1238/2476] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:13:59 2022] DELETE /identity/v3/projects/afb9f0f168b643cab83c9b8a393464af => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:00.078875 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ac06f-deb7-4293-b0de-d4c029c8fca3 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:00.079449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ac06f-deb7-4293-b0de-d4c029c8fca3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:00.079547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ac06f-deb7-4293-b0de-d4c029c8fca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:00.098830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-2d5ac06f-deb7-4293-b0de-d4c029c8fca3 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:00.100078 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2d5ac06f-deb7-4293-b0de-d4c029c8fca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:00.144453 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:00.165493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1239/2477] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:14:00.173851 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5d634-304b-45c5-bbf6-baee823bbd66 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:00.175020 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5d634-304b-45c5-bbf6-baee823bbd66 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:00.175311 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5d634-304b-45c5-bbf6-baee823bbd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:00.195446 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f5c5d634-304b-45c5-bbf6-baee823bbd66 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:00.197221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f5c5d634-304b-45c5-bbf6-baee823bbd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:00.206849 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1239/2478] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:00.211145 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ac06f-deb7-4293-b0de-d4c029c8fca3 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:00.221373 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ac06f-deb7-4293-b0de-d4c029c8fca3 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:00.226267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd64ff4-227e-43c4-a05d-4302bcc7d92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:00.227138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd64ff4-227e-43c4-a05d-4302bcc7d92c admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:00.227832 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd64ff4-227e-43c4-a05d-4302bcc7d92c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:00.227832 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd64ff4-227e-43c4-a05d-4302bcc7d92c admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:00.228595 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd64ff4-227e-43c4-a05d-4302bcc7d92c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:14:00.229032 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd64ff4-227e-43c4-a05d-4302bcc7d92c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:00.230687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd64ff4-227e-43c4-a05d-4302bcc7d92c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:00.244109 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1240/2479] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 20:14:00.249003 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5d634-304b-45c5-bbf6-baee823bbd66 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:00.258251 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5d634-304b-45c5-bbf6-baee823bbd66 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:00.263219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516df279-65fd-44a3-a3d8-e336de201f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:00.264519 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516df279-65fd-44a3-a3d8-e336de201f93 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:00.264809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516df279-65fd-44a3-a3d8-e336de201f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:00.265087 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516df279-65fd-44a3-a3d8-e336de201f93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:00.265823 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516df279-65fd-44a3-a3d8-e336de201f93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:00.267779 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516df279-65fd-44a3-a3d8-e336de201f93 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:00.348030 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1240/2480] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:14:00 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:14:00.352194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd64ff4-227e-43c4-a05d-4302bcc7d92c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:00.363618 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd64ff4-227e-43c4-a05d-4302bcc7d92c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:00.423763 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4c8fc-2698-439d-9410-c0001be559b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:00.424474 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4c8fc-2698-439d-9410-c0001be559b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:00.424756 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4c8fc-2698-439d-9410-c0001be559b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:00.425030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4c8fc-2698-439d-9410-c0001be559b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:00.425726 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df4c8fc-2698-439d-9410-c0001be559b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:00.427666 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df4c8fc-2698-439d-9410-c0001be559b5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:00.504191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1241/2481] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:14:00 2022] POST /identity/v3/users => generated 290 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:14:00.505213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516df279-65fd-44a3-a3d8-e336de201f93 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:00.515647 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516df279-65fd-44a3-a3d8-e336de201f93 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:00.520258 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b81f0-e6fb-494c-aa87-4f822a916eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:00.520932 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b81f0-e6fb-494c-aa87-4f822a916eae admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:00.523210 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b81f0-e6fb-494c-aa87-4f822a916eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:00.523210 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b81f0-e6fb-494c-aa87-4f822a916eae admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:00.523210 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9b81f0-e6fb-494c-aa87-4f822a916eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:00.523757 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9b81f0-e6fb-494c-aa87-4f822a916eae admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:00.533998 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1241/2482] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:14:00.538084 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4c8fc-2698-439d-9410-c0001be559b5 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:00.548973 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4c8fc-2698-439d-9410-c0001be559b5 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:00.553639 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf65943-9591-42f4-8e37-4139398055f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:00.554294 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf65943-9591-42f4-8e37-4139398055f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:00.554579 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf65943-9591-42f4-8e37-4139398055f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:00.554856 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf65943-9591-42f4-8e37-4139398055f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:00.555610 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf65943-9591-42f4-8e37-4139398055f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:00.557310 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf65943-9591-42f4-8e37-4139398055f4 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:00.571325 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1242/2483] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:14:00.572539 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b81f0-e6fb-494c-aa87-4f822a916eae admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:00.586718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b81f0-e6fb-494c-aa87-4f822a916eae admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:00.591650 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc2be6-5c65-40a4-a2cd-cab1105fec48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:00.592592 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc2be6-5c65-40a4-a2cd-cab1105fec48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:00.592874 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc2be6-5c65-40a4-a2cd-cab1105fec48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:00.593145 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc2be6-5c65-40a4-a2cd-cab1105fec48 admin admin] PATH_INFO: `/v3/projects/6d8ef8f9fcf24e2c841a5186ebad69d9/users/7412adcbe386495db31fcdfb8bf3004e/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:00.611417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc2be6-5c65-40a4-a2cd-cab1105fec48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d8ef8f9fcf24e2c841a5186ebad69d9, user_id=7412adcbe386495db31fcdfb8bf3004e, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:00.614100 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc2be6-5c65-40a4-a2cd-cab1105fec48 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:00.692833 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1242/2484] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:14:00 2022] PUT /identity/v3/projects/6d8ef8f9fcf24e2c841a5186ebad69d9/users/7412adcbe386495db31fcdfb8bf3004e/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:00.703175 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da453758-5172-4e96-a211-780096a04b33 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:00.703592 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da453758-5172-4e96-a211-780096a04b33 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:00.703875 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da453758-5172-4e96-a211-780096a04b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:00.733709 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-da453758-5172-4e96-a211-780096a04b33 None None] MFA Rules not processed for user `7412adcbe386495db31fcdfb8bf3004e`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:00.735300 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-da453758-5172-4e96-a211-780096a04b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:00.807001 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1243/2485] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:00.811492 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc2be6-5c65-40a4-a2cd-cab1105fec48 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:00.861018 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:00.874580 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc2be6-5c65-40a4-a2cd-cab1105fec48 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:00.879632 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a4136-b4c3-4d2d-9f0b-c907b57864a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:00.880446 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a4136-b4c3-4d2d-9f0b-c907b57864a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:00.880836 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a4136-b4c3-4d2d-9f0b-c907b57864a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:00.881373 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a4136-b4c3-4d2d-9f0b-c907b57864a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:00.882194 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a4136-b4c3-4d2d-9f0b-c907b57864a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:00.884223 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a4136-b4c3-4d2d-9f0b-c907b57864a0 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:00.979433 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1243/2486] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:14:00 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:14:00.982582 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da453758-5172-4e96-a211-780096a04b33 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:00.993638 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da453758-5172-4e96-a211-780096a04b33 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:01.030501 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:01.048446 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df85684-5701-426a-89fa-44df412635d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:01.049110 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df85684-5701-426a-89fa-44df412635d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:01.049389 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df85684-5701-426a-89fa-44df412635d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:01.049662 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df85684-5701-426a-89fa-44df412635d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:01.050343 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df85684-5701-426a-89fa-44df412635d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:01.052266 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df85684-5701-426a-89fa-44df412635d2 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:01.159390 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1244/2487] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:14:00 2022] POST /identity/v3/users => generated 296 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:14:01.160606 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a4136-b4c3-4d2d-9f0b-c907b57864a0 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:01.170822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a4136-b4c3-4d2d-9f0b-c907b57864a0 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:01.175499 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782be884-307a-4c23-b7e0-5b46a038ada1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:01.176160 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782be884-307a-4c23-b7e0-5b46a038ada1 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:01.176479 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782be884-307a-4c23-b7e0-5b46a038ada1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:01.176755 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782be884-307a-4c23-b7e0-5b46a038ada1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:01.177510 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782be884-307a-4c23-b7e0-5b46a038ada1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:01.179102 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782be884-307a-4c23-b7e0-5b46a038ada1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:01.190630 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1244/2488] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:14:01.194831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df85684-5701-426a-89fa-44df412635d2 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:01.205969 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df85684-5701-426a-89fa-44df412635d2 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:01.210872 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40aecd-6902-4989-a64a-4b6c35344fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:01.211739 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40aecd-6902-4989-a64a-4b6c35344fb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:01.212025 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40aecd-6902-4989-a64a-4b6c35344fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:01.212318 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40aecd-6902-4989-a64a-4b6c35344fb4 admin admin] PATH_INFO: `/v3/projects/336aadb30ba042cdbd008ba0d4b910ab/users/f7efae875b2341848f8c3a60d37091ed/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:01.228501 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40aecd-6902-4989-a64a-4b6c35344fb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=336aadb30ba042cdbd008ba0d4b910ab, user_id=f7efae875b2341848f8c3a60d37091ed, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:01.231185 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40aecd-6902-4989-a64a-4b6c35344fb4 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:01.317876 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1245/2489] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:14:01 2022] PUT /identity/v3/projects/336aadb30ba042cdbd008ba0d4b910ab/users/f7efae875b2341848f8c3a60d37091ed/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:01.321828 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782be884-307a-4c23-b7e0-5b46a038ada1 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:01.332675 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782be884-307a-4c23-b7e0-5b46a038ada1 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:01.393151 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea52efa-d514-43f9-93d3-b24139a21791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:01.393743 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea52efa-d514-43f9-93d3-b24139a21791 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:01.394031 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea52efa-d514-43f9-93d3-b24139a21791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:01.394313 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea52efa-d514-43f9-93d3-b24139a21791 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:01.395043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea52efa-d514-43f9-93d3-b24139a21791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:01.396707 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea52efa-d514-43f9-93d3-b24139a21791 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:01.408490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1245/2490] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:14:01.412558 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40aecd-6902-4989-a64a-4b6c35344fb4 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:01.424051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40aecd-6902-4989-a64a-4b6c35344fb4 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:01.428889 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97edd2-a071-40a4-8262-5a286281be8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:01.429780 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97edd2-a071-40a4-8262-5a286281be8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:01.430063 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97edd2-a071-40a4-8262-5a286281be8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:01.430335 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97edd2-a071-40a4-8262-5a286281be8b admin admin] PATH_INFO: `/v3/projects/336aadb30ba042cdbd008ba0d4b910ab/users/f7efae875b2341848f8c3a60d37091ed/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:01.432348 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c97edd2-a071-40a4-8262-5a286281be8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=336aadb30ba042cdbd008ba0d4b910ab, user_id=f7efae875b2341848f8c3a60d37091ed, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:01.435014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c97edd2-a071-40a4-8262-5a286281be8b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:01.445806 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1246/2491] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:14:01 2022] PUT /identity/v3/projects/336aadb30ba042cdbd008ba0d4b910ab/users/f7efae875b2341848f8c3a60d37091ed/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:14:01.459469 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c87c63-8ae0-427c-8ec3-0c8d477e7c0a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:01.459827 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c87c63-8ae0-427c-8ec3-0c8d477e7c0a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:01.460124 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c87c63-8ae0-427c-8ec3-0c8d477e7c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:01.487702 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-01c87c63-8ae0-427c-8ec3-0c8d477e7c0a None None] MFA Rules not processed for user `f7efae875b2341848f8c3a60d37091ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:01.492568 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-01c87c63-8ae0-427c-8ec3-0c8d477e7c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:01.561931 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1246/2492] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:01 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:01.572632 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97edd2-a071-40a4-8262-5a286281be8b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:01.574346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3c97edd2-a071-40a4-8262-5a286281be8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:01.632001 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97edd2-a071-40a4-8262-5a286281be8b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:01.635743 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6883d117-73bb-4c4c-9a55-2b5790e7af89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:01.636417 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883d117-73bb-4c4c-9a55-2b5790e7af89 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:01.636699 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883d117-73bb-4c4c-9a55-2b5790e7af89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:01.636971 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6883d117-73bb-4c4c-9a55-2b5790e7af89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:01.647480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883d117-73bb-4c4c-9a55-2b5790e7af89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:01.648968 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6883d117-73bb-4c4c-9a55-2b5790e7af89 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:01.658861 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1247/2493] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:14:01 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:14:02.206703 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c87c63-8ae0-427c-8ec3-0c8d477e7c0a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:02.259196 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:02.275318 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c87c63-8ae0-427c-8ec3-0c8d477e7c0a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:02.280455 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09cdbcf-52e0-4806-ba07-be1befdd4414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:02.281331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cdbcf-52e0-4806-ba07-be1befdd4414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:02.281619 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cdbcf-52e0-4806-ba07-be1befdd4414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:02.281895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cdbcf-52e0-4806-ba07-be1befdd4414 admin admin] PATH_INFO: `/v3/users/7412adcbe386495db31fcdfb8bf3004e` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:02.283091 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09cdbcf-52e0-4806-ba07-be1befdd4414 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7412adcbe386495db31fcdfb8bf3004e)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:02.285069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09cdbcf-52e0-4806-ba07-be1befdd4414 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:02.493871 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f09cdbcf-52e0-4806-ba07-be1befdd4414 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7412adcbe386495db31fcdfb8bf3004e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:02.494701 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f09cdbcf-52e0-4806-ba07-be1befdd4414 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7412adcbe386495db31fcdfb8bf3004e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:02.516151 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f09cdbcf-52e0-4806-ba07-be1befdd4414 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7412adcbe386495db31fcdfb8bf3004e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:02.525242 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f09cdbcf-52e0-4806-ba07-be1befdd4414 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7412adcbe386495db31fcdfb8bf3004e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:02.580634 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f09cdbcf-52e0-4806-ba07-be1befdd4414 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7412adcbe386495db31fcdfb8bf3004e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:02.588407 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1247/2494] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:14:02 2022] DELETE /identity/v3/users/7412adcbe386495db31fcdfb8bf3004e => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:02.593675 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406596c-338c-40d5-95f5-1250306d78a5 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:02.594199 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406596c-338c-40d5-95f5-1250306d78a5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:02.594477 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c406596c-338c-40d5-95f5-1250306d78a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:02.613172 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c406596c-338c-40d5-95f5-1250306d78a5 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:02.614486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c406596c-338c-40d5-95f5-1250306d78a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:02.675425 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1248/2495] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:02.688748 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09cdbcf-52e0-4806-ba07-be1befdd4414 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:02.690807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f09cdbcf-52e0-4806-ba07-be1befdd4414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:02.755759 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09cdbcf-52e0-4806-ba07-be1befdd4414 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:02.759395 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73557ee-f66e-43d9-8cd0-dfbd2f4ee506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:02.760041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73557ee-f66e-43d9-8cd0-dfbd2f4ee506 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:02.760355 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73557ee-f66e-43d9-8cd0-dfbd2f4ee506 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:02.760638 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73557ee-f66e-43d9-8cd0-dfbd2f4ee506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:02.772648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73557ee-f66e-43d9-8cd0-dfbd2f4ee506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:02.774535 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73557ee-f66e-43d9-8cd0-dfbd2f4ee506 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:02.786509 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1248/2496] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:14:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:14:03.280376 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406596c-338c-40d5-95f5-1250306d78a5 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:03.290809 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c406596c-338c-40d5-95f5-1250306d78a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:03.316724 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c406596c-338c-40d5-95f5-1250306d78a5 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:03.324055 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a91291-6733-4759-bc4b-3ba8cd04be87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:03.324972 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a91291-6733-4759-bc4b-3ba8cd04be87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:03.325253 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a91291-6733-4759-bc4b-3ba8cd04be87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:03.325547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a91291-6733-4759-bc4b-3ba8cd04be87 admin admin] PATH_INFO: `/v3/projects/6d8ef8f9fcf24e2c841a5186ebad69d9` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:03.336806 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a91291-6733-4759-bc4b-3ba8cd04be87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d8ef8f9fcf24e2c841a5186ebad69d9)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:03.336806 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a91291-6733-4759-bc4b-3ba8cd04be87 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:03.418546 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:03.432890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-49a91291-6733-4759-bc4b-3ba8cd04be87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d8ef8f9fcf24e2c841a5186ebad69d9', 'request_id': 'req-49a91291-6733-4759-bc4b-3ba8cd04be87'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:03.490049 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-49a91291-6733-4759-bc4b-3ba8cd04be87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d8ef8f9fcf24e2c841a5186ebad69d9', 'request_id': 'req-49a91291-6733-4759-bc4b-3ba8cd04be87'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:03.500022 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-49a91291-6733-4759-bc4b-3ba8cd04be87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d8ef8f9fcf24e2c841a5186ebad69d9', 'request_id': 'req-49a91291-6733-4759-bc4b-3ba8cd04be87'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:03.512487 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-49a91291-6733-4759-bc4b-3ba8cd04be87 admin admin] The token cache is being invalidate because project 6d8ef8f9fcf24e2c841a5186ebad69d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:14:03.512743 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-49a91291-6733-4759-bc4b-3ba8cd04be87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:03.513495 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-49a91291-6733-4759-bc4b-3ba8cd04be87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:03.515261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1249/2497] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:14:03 2022] DELETE /identity/v3/projects/6d8ef8f9fcf24e2c841a5186ebad69d9 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:03.519208 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73557ee-f66e-43d9-8cd0-dfbd2f4ee506 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:03.529769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73557ee-f66e-43d9-8cd0-dfbd2f4ee506 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:03.593142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e2c8c-314c-4c29-9665-c3e811f80ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:03.594107 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e2c8c-314c-4c29-9665-c3e811f80ec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:03.594388 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e2c8c-314c-4c29-9665-c3e811f80ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:03.594663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e2c8c-314c-4c29-9665-c3e811f80ec5 admin admin] PATH_INFO: `/v3/users/f7efae875b2341848f8c3a60d37091ed` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:03.595833 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3e2c8c-314c-4c29-9665-c3e811f80ec5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7efae875b2341848f8c3a60d37091ed)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:03.597872 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3e2c8c-314c-4c29-9665-c3e811f80ec5 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:03.604502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:03.780742 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9b3e2c8c-314c-4c29-9665-c3e811f80ec5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7efae875b2341848f8c3a60d37091ed'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:03.781492 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9b3e2c8c-314c-4c29-9665-c3e811f80ec5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7efae875b2341848f8c3a60d37091ed'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:03.793616 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9b3e2c8c-314c-4c29-9665-c3e811f80ec5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7efae875b2341848f8c3a60d37091ed'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:03.801122 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9b3e2c8c-314c-4c29-9665-c3e811f80ec5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7efae875b2341848f8c3a60d37091ed'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:03.888932 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9b3e2c8c-314c-4c29-9665-c3e811f80ec5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7efae875b2341848f8c3a60d37091ed'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:03.895358 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1249/2498] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:14:03 2022] DELETE /identity/v3/users/f7efae875b2341848f8c3a60d37091ed => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:04.332289 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a91291-6733-4759-bc4b-3ba8cd04be87 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:04.333937 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-49a91291-6733-4759-bc4b-3ba8cd04be87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:04.348918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a91291-6733-4759-bc4b-3ba8cd04be87 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:04.406985 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea4518-6664-4830-8d26-bb6bacc67493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:04.407909 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea4518-6664-4830-8d26-bb6bacc67493 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:04.408194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea4518-6664-4830-8d26-bb6bacc67493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:04.408506 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea4518-6664-4830-8d26-bb6bacc67493 admin admin] PATH_INFO: `/v3/projects/336aadb30ba042cdbd008ba0d4b910ab` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:04.409736 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ea4518-6664-4830-8d26-bb6bacc67493 admin admin] RBAC: Authorizing `identity:delete_project(project_id=336aadb30ba042cdbd008ba0d4b910ab)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:04.411710 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ea4518-6664-4830-8d26-bb6bacc67493 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:04.512547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-70ea4518-6664-4830-8d26-bb6bacc67493 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '336aadb30ba042cdbd008ba0d4b910ab', 'request_id': 'req-70ea4518-6664-4830-8d26-bb6bacc67493'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:04.613994 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-70ea4518-6664-4830-8d26-bb6bacc67493 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '336aadb30ba042cdbd008ba0d4b910ab', 'request_id': 'req-70ea4518-6664-4830-8d26-bb6bacc67493'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:04.619855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-70ea4518-6664-4830-8d26-bb6bacc67493 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '336aadb30ba042cdbd008ba0d4b910ab', 'request_id': 'req-70ea4518-6664-4830-8d26-bb6bacc67493'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:04.631710 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-70ea4518-6664-4830-8d26-bb6bacc67493 admin admin] The token cache is being invalidate because project 336aadb30ba042cdbd008ba0d4b910ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:14:04.632025 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-70ea4518-6664-4830-8d26-bb6bacc67493 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:04.632734 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-70ea4518-6664-4830-8d26-bb6bacc67493 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:04.634258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1250/2499] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:14:04 2022] DELETE /identity/v3/projects/336aadb30ba042cdbd008ba0d4b910ab => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:04.640201 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d14986-1185-48f2-b956-afefa8312168 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:04.640714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d14986-1185-48f2-b956-afefa8312168 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:04.640809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d14986-1185-48f2-b956-afefa8312168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:04.658775 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c5d14986-1185-48f2-b956-afefa8312168 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:04.659975 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c5d14986-1185-48f2-b956-afefa8312168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:04.721072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1250/2500] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:04.728771 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd2857-3db4-4109-abb4-5f23ee53e91d None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:04.729049 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd2857-3db4-4109-abb4-5f23ee53e91d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:04.729362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd2857-3db4-4109-abb4-5f23ee53e91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:04.750523 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ccbd2857-3db4-4109-abb4-5f23ee53e91d None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:04.751851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ccbd2857-3db4-4109-abb4-5f23ee53e91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:04.760915 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1251/2501] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:04.766155 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d14986-1185-48f2-b956-afefa8312168 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:04.777030 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d14986-1185-48f2-b956-afefa8312168 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:04.782892 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e200d-2fe1-4f51-a3ae-ea47b2a8a23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:04.783771 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e200d-2fe1-4f51-a3ae-ea47b2a8a23f admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:04.784331 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e200d-2fe1-4f51-a3ae-ea47b2a8a23f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:04.784908 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e200d-2fe1-4f51-a3ae-ea47b2a8a23f admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:04.785765 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e200d-2fe1-4f51-a3ae-ea47b2a8a23f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:14:04.786081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e200d-2fe1-4f51-a3ae-ea47b2a8a23f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:04.789020 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e200d-2fe1-4f51-a3ae-ea47b2a8a23f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:04.802285 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1251/2502] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:14:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:14:04.806689 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd2857-3db4-4109-abb4-5f23ee53e91d None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:04.817029 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd2857-3db4-4109-abb4-5f23ee53e91d None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:04.821649 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350b017-692d-48e4-9bc4-e3841aa26cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:04.822289 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350b017-692d-48e4-9bc4-e3841aa26cdd admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:04.822566 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350b017-692d-48e4-9bc4-e3841aa26cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:04.822836 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350b017-692d-48e4-9bc4-e3841aa26cdd admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:04.823525 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5350b017-692d-48e4-9bc4-e3841aa26cdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:04.825515 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5350b017-692d-48e4-9bc4-e3841aa26cdd admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:04.936230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1252/2503] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:14:04 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:14:04.940286 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e200d-2fe1-4f51-a3ae-ea47b2a8a23f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:04.950267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e200d-2fe1-4f51-a3ae-ea47b2a8a23f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:05.006311 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2faf3aa-6c3a-4b75-9bbb-1716281c96e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:05.007066 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2faf3aa-6c3a-4b75-9bbb-1716281c96e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:05.007348 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2faf3aa-6c3a-4b75-9bbb-1716281c96e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:05.007629 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2faf3aa-6c3a-4b75-9bbb-1716281c96e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:05.008433 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2faf3aa-6c3a-4b75-9bbb-1716281c96e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:05.010441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2faf3aa-6c3a-4b75-9bbb-1716281c96e9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:05.103570 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1252/2504] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:14:04 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:14:05.108287 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350b017-692d-48e4-9bc4-e3841aa26cdd admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:05.118504 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350b017-692d-48e4-9bc4-e3841aa26cdd admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:05.123639 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417e06c-856d-4784-9baf-591551e429c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:05.124281 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417e06c-856d-4784-9baf-591551e429c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:05.124589 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417e06c-856d-4784-9baf-591551e429c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:05.124881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417e06c-856d-4784-9baf-591551e429c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:05.125603 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1417e06c-856d-4784-9baf-591551e429c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:05.127215 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1417e06c-856d-4784-9baf-591551e429c0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:05.137690 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1253/2505] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:14:05.141615 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2faf3aa-6c3a-4b75-9bbb-1716281c96e9 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:05.146770 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:05.152049 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2faf3aa-6c3a-4b75-9bbb-1716281c96e9 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:05.156891 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd233a9e-7361-4b43-b5ee-3b15271eab25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:05.157544 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd233a9e-7361-4b43-b5ee-3b15271eab25 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:05.157828 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd233a9e-7361-4b43-b5ee-3b15271eab25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:05.158099 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd233a9e-7361-4b43-b5ee-3b15271eab25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:05.158816 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd233a9e-7361-4b43-b5ee-3b15271eab25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:05.160457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd233a9e-7361-4b43-b5ee-3b15271eab25 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:05.171493 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1253/2506] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:14:05.175726 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417e06c-856d-4784-9baf-591551e429c0 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:05.194019 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417e06c-856d-4784-9baf-591551e429c0 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:05.196173 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f188547-892e-4a82-9646-52560e66f4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:05.197074 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f188547-892e-4a82-9646-52560e66f4d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:05.197493 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f188547-892e-4a82-9646-52560e66f4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:05.197906 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f188547-892e-4a82-9646-52560e66f4d5 admin admin] PATH_INFO: `/v3/projects/11f0e7a28d424c3795f86b2ba0b5e8d7/users/9abd2ef0824b4f0fbd710ff458ecc22d/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:05.201552 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:05.213039 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f188547-892e-4a82-9646-52560e66f4d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11f0e7a28d424c3795f86b2ba0b5e8d7, user_id=9abd2ef0824b4f0fbd710ff458ecc22d, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:05.215742 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f188547-892e-4a82-9646-52560e66f4d5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:05.281276 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1254/2507] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:14:05 2022] PUT /identity/v3/projects/11f0e7a28d424c3795f86b2ba0b5e8d7/users/9abd2ef0824b4f0fbd710ff458ecc22d/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:05.291251 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ac6d6-ff00-4ef4-83e4-3a63aaacdac9 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:05.291542 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ac6d6-ff00-4ef4-83e4-3a63aaacdac9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:05.291847 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ac6d6-ff00-4ef4-83e4-3a63aaacdac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:05.317954 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-847ac6d6-ff00-4ef4-83e4-3a63aaacdac9 None None] MFA Rules not processed for user `9abd2ef0824b4f0fbd710ff458ecc22d`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:05.319157 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-847ac6d6-ff00-4ef4-83e4-3a63aaacdac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:05.383247 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1254/2508] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:05.394254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f188547-892e-4a82-9646-52560e66f4d5 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:05.395935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4f188547-892e-4a82-9646-52560e66f4d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:05.468756 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f188547-892e-4a82-9646-52560e66f4d5 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:05.472420 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff5351-ed3e-4462-b9eb-72e4a434c1c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:05.473067 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff5351-ed3e-4462-b9eb-72e4a434c1c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:05.473346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff5351-ed3e-4462-b9eb-72e4a434c1c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:05.473616 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff5351-ed3e-4462-b9eb-72e4a434c1c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:05.483652 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ff5351-ed3e-4462-b9eb-72e4a434c1c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:05.485135 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ff5351-ed3e-4462-b9eb-72e4a434c1c8 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:05.494049 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1255/2509] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:14:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:14:05.513831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ac6d6-ff00-4ef4-83e4-3a63aaacdac9 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:05.575696 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ac6d6-ff00-4ef4-83e4-3a63aaacdac9 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:05.580343 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32846ec6-82da-44f2-848a-cdef0f589118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:05.581479 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32846ec6-82da-44f2-848a-cdef0f589118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:05.581754 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32846ec6-82da-44f2-848a-cdef0f589118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:05.582021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32846ec6-82da-44f2-848a-cdef0f589118 admin admin] PATH_INFO: `/v3/users/9abd2ef0824b4f0fbd710ff458ecc22d` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:05.583221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32846ec6-82da-44f2-848a-cdef0f589118 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9abd2ef0824b4f0fbd710ff458ecc22d)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:05.585181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32846ec6-82da-44f2-848a-cdef0f589118 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:05.776782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-32846ec6-82da-44f2-848a-cdef0f589118 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9abd2ef0824b4f0fbd710ff458ecc22d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:05.777436 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-32846ec6-82da-44f2-848a-cdef0f589118 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9abd2ef0824b4f0fbd710ff458ecc22d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:05.790030 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-32846ec6-82da-44f2-848a-cdef0f589118 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9abd2ef0824b4f0fbd710ff458ecc22d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:05.796885 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-32846ec6-82da-44f2-848a-cdef0f589118 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9abd2ef0824b4f0fbd710ff458ecc22d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:05.857507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-32846ec6-82da-44f2-848a-cdef0f589118 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9abd2ef0824b4f0fbd710ff458ecc22d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:05.859873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:05.864263 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1255/2510] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:14:05 2022] DELETE /identity/v3/users/9abd2ef0824b4f0fbd710ff458ecc22d => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:14:05.868733 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c89ff2-91cc-46f6-9770-a94aa2138487 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:05.869018 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c89ff2-91cc-46f6-9770-a94aa2138487 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:05.874682 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c89ff2-91cc-46f6-9770-a94aa2138487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:05.885919 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-41c89ff2-91cc-46f6-9770-a94aa2138487 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:05.887122 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-41c89ff2-91cc-46f6-9770-a94aa2138487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:05.946237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1256/2511] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:05.956695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32846ec6-82da-44f2-848a-cdef0f589118 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:05.958760 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-32846ec6-82da-44f2-848a-cdef0f589118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:06.019946 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32846ec6-82da-44f2-848a-cdef0f589118 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:06.024505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190bfb4-5d4b-4448-9a82-18a0dec5210f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:06.025114 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190bfb4-5d4b-4448-9a82-18a0dec5210f service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:06.025396 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190bfb4-5d4b-4448-9a82-18a0dec5210f service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:06.025669 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190bfb4-5d4b-4448-9a82-18a0dec5210f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:06.037447 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190bfb4-5d4b-4448-9a82-18a0dec5210f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:06.038893 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190bfb4-5d4b-4448-9a82-18a0dec5210f service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:06.049730 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1256/2512] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:14:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:14:06.539146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c89ff2-91cc-46f6-9770-a94aa2138487 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:06.540646 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-41c89ff2-91cc-46f6-9770-a94aa2138487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:06.554615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c89ff2-91cc-46f6-9770-a94aa2138487 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:06.558883 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8bc4ef-f4b7-4ac4-b586-fc22eacb9fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:06.559836 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8bc4ef-f4b7-4ac4-b586-fc22eacb9fad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:06.560786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8bc4ef-f4b7-4ac4-b586-fc22eacb9fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:06.561061 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8bc4ef-f4b7-4ac4-b586-fc22eacb9fad admin admin] PATH_INFO: `/v3/projects/11f0e7a28d424c3795f86b2ba0b5e8d7` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:06.562375 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8bc4ef-f4b7-4ac4-b586-fc22eacb9fad admin admin] RBAC: Authorizing `identity:delete_project(project_id=11f0e7a28d424c3795f86b2ba0b5e8d7)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:06.564429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8bc4ef-f4b7-4ac4-b586-fc22eacb9fad admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:06.710342 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0d8bc4ef-f4b7-4ac4-b586-fc22eacb9fad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11f0e7a28d424c3795f86b2ba0b5e8d7', 'request_id': 'req-0d8bc4ef-f4b7-4ac4-b586-fc22eacb9fad'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:06.800964 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0d8bc4ef-f4b7-4ac4-b586-fc22eacb9fad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11f0e7a28d424c3795f86b2ba0b5e8d7', 'request_id': 'req-0d8bc4ef-f4b7-4ac4-b586-fc22eacb9fad'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:06.811204 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0d8bc4ef-f4b7-4ac4-b586-fc22eacb9fad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11f0e7a28d424c3795f86b2ba0b5e8d7', 'request_id': 'req-0d8bc4ef-f4b7-4ac4-b586-fc22eacb9fad'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:06.837396 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0d8bc4ef-f4b7-4ac4-b586-fc22eacb9fad admin admin] The token cache is being invalidate because project 11f0e7a28d424c3795f86b2ba0b5e8d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:14:06.837667 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0d8bc4ef-f4b7-4ac4-b586-fc22eacb9fad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:06.838323 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0d8bc4ef-f4b7-4ac4-b586-fc22eacb9fad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:06.839927 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1257/2513] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:14:06 2022] DELETE /identity/v3/projects/11f0e7a28d424c3795f86b2ba0b5e8d7 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:06.848726 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b71fa9-9a76-4bf0-8e65-7d0f39accad7 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:06.849335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b71fa9-9a76-4bf0-8e65-7d0f39accad7 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:06.849335 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b71fa9-9a76-4bf0-8e65-7d0f39accad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:06.869935 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8bc4ef-f4b7-4ac4-b586-fc22eacb9fad admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:06.870920 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b6b71fa9-9a76-4bf0-8e65-7d0f39accad7 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:06.871415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0d8bc4ef-f4b7-4ac4-b586-fc22eacb9fad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:06.872154 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b6b71fa9-9a76-4bf0-8e65-7d0f39accad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:06.886186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:06.941694 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8bc4ef-f4b7-4ac4-b586-fc22eacb9fad admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:06.945498 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd717a-635c-4d97-b234-411df2fd3ac2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:06.946145 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd717a-635c-4d97-b234-411df2fd3ac2 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:06.946430 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd717a-635c-4d97-b234-411df2fd3ac2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:06.946713 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbd717a-635c-4d97-b234-411df2fd3ac2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:06.947292 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1257/2514] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:06.948901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fcbd717a-635c-4d97-b234-411df2fd3ac2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:06.955119 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1826a9-76c1-4376-af62-dfb8d709b240 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:06.955408 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1826a9-76c1-4376-af62-dfb8d709b240 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:06.955689 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1826a9-76c1-4376-af62-dfb8d709b240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:06.961886 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbd717a-635c-4d97-b234-411df2fd3ac2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:06.963333 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbd717a-635c-4d97-b234-411df2fd3ac2 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:06.973783 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1258/2515] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:14:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 20:14:06.976264 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7b1826a9-76c1-4376-af62-dfb8d709b240 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:06.977939 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7b1826a9-76c1-4376-af62-dfb8d709b240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:06.986855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1258/2516] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:14:06.990979 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd717a-635c-4d97-b234-411df2fd3ac2 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:07.001150 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbd717a-635c-4d97-b234-411df2fd3ac2 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:07.005967 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d19198-bde8-4e55-b58f-0ebd136ef28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:07.006848 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d19198-bde8-4e55-b58f-0ebd136ef28d admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:07.007175 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d19198-bde8-4e55-b58f-0ebd136ef28d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:07.007941 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d19198-bde8-4e55-b58f-0ebd136ef28d admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:07.008850 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d19198-bde8-4e55-b58f-0ebd136ef28d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:14:07.009167 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d19198-bde8-4e55-b58f-0ebd136ef28d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:07.010787 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d19198-bde8-4e55-b58f-0ebd136ef28d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:07.022572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1259/2517] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:14:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:14:07.026956 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1826a9-76c1-4376-af62-dfb8d709b240 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:07.036858 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1826a9-76c1-4376-af62-dfb8d709b240 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:07.041383 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7888d-0d6b-4744-9f72-7929047a7cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:07.042030 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7888d-0d6b-4744-9f72-7929047a7cad admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:07.042304 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7888d-0d6b-4744-9f72-7929047a7cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:07.042573 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7888d-0d6b-4744-9f72-7929047a7cad admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:07.043260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a7888d-0d6b-4744-9f72-7929047a7cad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:07.045856 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a7888d-0d6b-4744-9f72-7929047a7cad admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:07.164858 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1259/2518] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:14:07 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:14:07.168942 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d19198-bde8-4e55-b58f-0ebd136ef28d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:07.178872 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d19198-bde8-4e55-b58f-0ebd136ef28d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:07.246479 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c4867-c24b-417b-bc98-aa566447e514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:07.247200 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c4867-c24b-417b-bc98-aa566447e514 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:07.247563 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c4867-c24b-417b-bc98-aa566447e514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:07.247852 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c4867-c24b-417b-bc98-aa566447e514 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:07.248587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c4867-c24b-417b-bc98-aa566447e514 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:07.250564 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c4867-c24b-417b-bc98-aa566447e514 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:07.323502 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1260/2519] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:14:07 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:14:07.328173 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7888d-0d6b-4744-9f72-7929047a7cad admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:07.338763 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7888d-0d6b-4744-9f72-7929047a7cad admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:07.344024 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c69e60-52c8-4a53-a14a-0af3cac0c147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:07.363125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c69e60-52c8-4a53-a14a-0af3cac0c147 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:07.363125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c69e60-52c8-4a53-a14a-0af3cac0c147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:07.363125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c69e60-52c8-4a53-a14a-0af3cac0c147 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:07.363125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c69e60-52c8-4a53-a14a-0af3cac0c147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:07.363125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c69e60-52c8-4a53-a14a-0af3cac0c147 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:07.363125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1260/2520] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:14:07.366121 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c4867-c24b-417b-bc98-aa566447e514 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:07.376605 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c4867-c24b-417b-bc98-aa566447e514 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:07.381580 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7994e7-73f5-4009-b5c0-f29d2aa6ad81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:07.382236 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7994e7-73f5-4009-b5c0-f29d2aa6ad81 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:07.382522 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7994e7-73f5-4009-b5c0-f29d2aa6ad81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:07.382799 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7994e7-73f5-4009-b5c0-f29d2aa6ad81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:07.383648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7994e7-73f5-4009-b5c0-f29d2aa6ad81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:07.385294 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7994e7-73f5-4009-b5c0-f29d2aa6ad81 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:07.399674 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1261/2521] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:14:07.400890 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c69e60-52c8-4a53-a14a-0af3cac0c147 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:07.411529 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c69e60-52c8-4a53-a14a-0af3cac0c147 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:07.417076 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7695eeb-35f6-4c96-be9c-a6f2de327fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:07.417953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7695eeb-35f6-4c96-be9c-a6f2de327fad admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:07.418232 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7695eeb-35f6-4c96-be9c-a6f2de327fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:07.418504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7695eeb-35f6-4c96-be9c-a6f2de327fad admin admin] PATH_INFO: `/v3/projects/2c88db35d2f747fbbedc5b2dc4fecbc2/users/f42eaa94d7a547139b51503b24937ff7/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:07.433536 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7695eeb-35f6-4c96-be9c-a6f2de327fad admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c88db35d2f747fbbedc5b2dc4fecbc2, user_id=f42eaa94d7a547139b51503b24937ff7, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:07.436350 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7695eeb-35f6-4c96-be9c-a6f2de327fad admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:07.507104 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1261/2522] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:14:07 2022] PUT /identity/v3/projects/2c88db35d2f747fbbedc5b2dc4fecbc2/users/f42eaa94d7a547139b51503b24937ff7/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:14:07.518351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5dbc3-75d6-4a37-8a38-255a420a5714 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:07.518669 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5dbc3-75d6-4a37-8a38-255a420a5714 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:07.519303 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5dbc3-75d6-4a37-8a38-255a420a5714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:07.547347 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-65d5dbc3-75d6-4a37-8a38-255a420a5714 None None] MFA Rules not processed for user `f42eaa94d7a547139b51503b24937ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:07.548531 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-65d5dbc3-75d6-4a37-8a38-255a420a5714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:07.614407 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1262/2523] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:07.618902 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7695eeb-35f6-4c96-be9c-a6f2de327fad admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:07.662043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:07.680863 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7695eeb-35f6-4c96-be9c-a6f2de327fad admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:07.686207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624821f-256d-4094-b5fd-0cfc6945e7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:07.686859 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624821f-256d-4094-b5fd-0cfc6945e7d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:07.687139 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624821f-256d-4094-b5fd-0cfc6945e7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:07.687409 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624821f-256d-4094-b5fd-0cfc6945e7d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:07.688104 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624821f-256d-4094-b5fd-0cfc6945e7d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:07.690064 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624821f-256d-4094-b5fd-0cfc6945e7d6 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:07.793484 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1262/2524] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:14:07 2022] POST /identity/v3/projects => generated 385 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:14:07.797551 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5dbc3-75d6-4a37-8a38-255a420a5714 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:07.808262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5dbc3-75d6-4a37-8a38-255a420a5714 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:07.813346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:07.872250 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554bc274-4351-4ec7-8642-e0a4edc9aa44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:07.873954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bc274-4351-4ec7-8642-e0a4edc9aa44 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:07.874431 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bc274-4351-4ec7-8642-e0a4edc9aa44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:07.874893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bc274-4351-4ec7-8642-e0a4edc9aa44 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:07.876144 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554bc274-4351-4ec7-8642-e0a4edc9aa44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:07.878940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554bc274-4351-4ec7-8642-e0a4edc9aa44 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:07.972345 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1263/2525] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:14:07 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:14:07.973551 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624821f-256d-4094-b5fd-0cfc6945e7d6 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:07.986653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624821f-256d-4094-b5fd-0cfc6945e7d6 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:07.992867 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e982b25-ae45-4241-baf8-8cd6356a740c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:07.993894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e982b25-ae45-4241-baf8-8cd6356a740c admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:07.994185 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e982b25-ae45-4241-baf8-8cd6356a740c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:07.994671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e982b25-ae45-4241-baf8-8cd6356a740c admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:07.995487 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e982b25-ae45-4241-baf8-8cd6356a740c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:07.997237 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e982b25-ae45-4241-baf8-8cd6356a740c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:08.012516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1263/2526] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:14:08.014545 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554bc274-4351-4ec7-8642-e0a4edc9aa44 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:08.026977 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554bc274-4351-4ec7-8642-e0a4edc9aa44 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:08.032957 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faeedda-4a05-4945-8d22-2675e10c9524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:08.034331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faeedda-4a05-4945-8d22-2675e10c9524 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:08.034331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faeedda-4a05-4945-8d22-2675e10c9524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:08.034331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faeedda-4a05-4945-8d22-2675e10c9524 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:08.035124 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faeedda-4a05-4945-8d22-2675e10c9524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:08.036994 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faeedda-4a05-4945-8d22-2675e10c9524 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:08.048966 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1264/2527] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:14:08.053077 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e982b25-ae45-4241-baf8-8cd6356a740c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:08.067138 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e982b25-ae45-4241-baf8-8cd6356a740c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:08.072277 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d4bf1-7e6e-4d6b-a592-49d2fc8a6de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:08.073429 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d4bf1-7e6e-4d6b-a592-49d2fc8a6de3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:08.073851 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d4bf1-7e6e-4d6b-a592-49d2fc8a6de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:08.074131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d4bf1-7e6e-4d6b-a592-49d2fc8a6de3 admin admin] PATH_INFO: `/v3/projects/d8ae82748b114685a74497f005b9ca54/users/8a8ff70a673340f5bd2d942ff633e5b2/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:08.092125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d4bf1-7e6e-4d6b-a592-49d2fc8a6de3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8ae82748b114685a74497f005b9ca54, user_id=8a8ff70a673340f5bd2d942ff633e5b2, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:08.094843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d4bf1-7e6e-4d6b-a592-49d2fc8a6de3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:08.207239 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1264/2528] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:14:08 2022] PUT /identity/v3/projects/d8ae82748b114685a74497f005b9ca54/users/8a8ff70a673340f5bd2d942ff633e5b2/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:08.219385 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd9d0e-89b1-40f7-bbc4-f775b374b42b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:08.219804 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd9d0e-89b1-40f7-bbc4-f775b374b42b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:08.220172 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd9d0e-89b1-40f7-bbc4-f775b374b42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:08.254890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8afd9d0e-89b1-40f7-bbc4-f775b374b42b None None] MFA Rules not processed for user `8a8ff70a673340f5bd2d942ff633e5b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:08.256652 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8afd9d0e-89b1-40f7-bbc4-f775b374b42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:08.341078 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1265/2529] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:08.345935 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d4bf1-7e6e-4d6b-a592-49d2fc8a6de3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:08.420757 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d4bf1-7e6e-4d6b-a592-49d2fc8a6de3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:08.425736 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e26c7b-ce3b-4a97-bfcc-8553b48ae583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:08.426614 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e26c7b-ce3b-4a97-bfcc-8553b48ae583 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:08.426982 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e26c7b-ce3b-4a97-bfcc-8553b48ae583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:08.427334 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e26c7b-ce3b-4a97-bfcc-8553b48ae583 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:08.428513 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e26c7b-ce3b-4a97-bfcc-8553b48ae583 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:08.430511 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e26c7b-ce3b-4a97-bfcc-8553b48ae583 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:08.523466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1265/2530] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:14:08 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:14:08.524979 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd9d0e-89b1-40f7-bbc4-f775b374b42b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:08.537211 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd9d0e-89b1-40f7-bbc4-f775b374b42b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:08.606809 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a4352e-3e05-43b7-b419-366668093aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:08.608112 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4352e-3e05-43b7-b419-366668093aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:08.608446 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4352e-3e05-43b7-b419-366668093aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:08.608789 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4352e-3e05-43b7-b419-366668093aa5 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:08.609552 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a4352e-3e05-43b7-b419-366668093aa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:08.611649 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a4352e-3e05-43b7-b419-366668093aa5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:08.703728 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1266/2531] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:14:08 2022] POST /identity/v3/users => generated 299 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:14:08.707939 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e26c7b-ce3b-4a97-bfcc-8553b48ae583 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:08.720268 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e26c7b-ce3b-4a97-bfcc-8553b48ae583 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:08.726862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad79efc-407d-4592-abb1-1e36646dcd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:08.727139 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad79efc-407d-4592-abb1-1e36646dcd81 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:08.727486 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad79efc-407d-4592-abb1-1e36646dcd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:08.727828 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad79efc-407d-4592-abb1-1e36646dcd81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:08.734602 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad79efc-407d-4592-abb1-1e36646dcd81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:08.734602 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad79efc-407d-4592-abb1-1e36646dcd81 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:08.744864 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1266/2532] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:14:08.749057 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a4352e-3e05-43b7-b419-366668093aa5 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:08.760204 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a4352e-3e05-43b7-b419-366668093aa5 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:08.765952 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5e448-5754-4d96-85e9-8da2aafea31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:08.766838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5e448-5754-4d96-85e9-8da2aafea31d admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:08.767121 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5e448-5754-4d96-85e9-8da2aafea31d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:08.767395 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5e448-5754-4d96-85e9-8da2aafea31d admin admin] PATH_INFO: `/v3/projects/78f7863c23584d4aaba9046ec14d5c1a/users/eb5b6f1a015349389e8ae9591da174ff/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:08.770049 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:08.784067 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d5e448-5754-4d96-85e9-8da2aafea31d admin admin] RBAC: Authorizing `identity:create_grant(project_id=78f7863c23584d4aaba9046ec14d5c1a, user_id=eb5b6f1a015349389e8ae9591da174ff, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:08.786793 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d5e448-5754-4d96-85e9-8da2aafea31d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:08.893413 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1267/2533] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:14:08 2022] PUT /identity/v3/projects/78f7863c23584d4aaba9046ec14d5c1a/users/eb5b6f1a015349389e8ae9591da174ff/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:08.894277 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad79efc-407d-4592-abb1-1e36646dcd81 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:08.910099 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad79efc-407d-4592-abb1-1e36646dcd81 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:08.979196 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf989cf-57d3-4021-881b-e8e256fdabe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:08.980018 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf989cf-57d3-4021-881b-e8e256fdabe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:08.980437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf989cf-57d3-4021-881b-e8e256fdabe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:08.980822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf989cf-57d3-4021-881b-e8e256fdabe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:08.981702 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf989cf-57d3-4021-881b-e8e256fdabe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:08.983583 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf989cf-57d3-4021-881b-e8e256fdabe3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:08.989554 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:09.001274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5e448-5754-4d96-85e9-8da2aafea31d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:09.007993 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1267/2534] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:14:09.014054 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5e448-5754-4d96-85e9-8da2aafea31d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:09.018911 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5edd9e-3830-4726-b7b7-d15255d9f7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:09.019822 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5edd9e-3830-4726-b7b7-d15255d9f7c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:09.020131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5edd9e-3830-4726-b7b7-d15255d9f7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:09.020390 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5edd9e-3830-4726-b7b7-d15255d9f7c2 admin admin] PATH_INFO: `/v3/projects/78f7863c23584d4aaba9046ec14d5c1a/users/eb5b6f1a015349389e8ae9591da174ff/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:09.026303 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5edd9e-3830-4726-b7b7-d15255d9f7c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78f7863c23584d4aaba9046ec14d5c1a, user_id=eb5b6f1a015349389e8ae9591da174ff, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:09.029207 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5edd9e-3830-4726-b7b7-d15255d9f7c2 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:09.039974 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1268/2535] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:14:09 2022] PUT /identity/v3/projects/78f7863c23584d4aaba9046ec14d5c1a/users/eb5b6f1a015349389e8ae9591da174ff/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:09.049923 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f636d-2aa8-44e7-96d7-72568cd3fd7b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:09.052511 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f636d-2aa8-44e7-96d7-72568cd3fd7b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:09.052511 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f636d-2aa8-44e7-96d7-72568cd3fd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:09.077646 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-698f636d-2aa8-44e7-96d7-72568cd3fd7b None None] MFA Rules not processed for user `eb5b6f1a015349389e8ae9591da174ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:09.078835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-698f636d-2aa8-44e7-96d7-72568cd3fd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:09.148248 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1268/2536] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:09 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:09.161377 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5edd9e-3830-4726-b7b7-d15255d9f7c2 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:09.161377 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3f5edd9e-3830-4726-b7b7-d15255d9f7c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:09.222734 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5edd9e-3830-4726-b7b7-d15255d9f7c2 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:09.230978 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954a86d-a068-4494-b736-daf107fae954 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:09.231679 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954a86d-a068-4494-b736-daf107fae954 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:09.231994 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954a86d-a068-4494-b736-daf107fae954 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:09.232268 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954a86d-a068-4494-b736-daf107fae954 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:09.292549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954a86d-a068-4494-b736-daf107fae954 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:09.293970 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1954a86d-a068-4494-b736-daf107fae954 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:09.303846 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1269/2537] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:14:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:14:10.806356 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f636d-2aa8-44e7-96d7-72568cd3fd7b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:10.807865 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-698f636d-2aa8-44e7-96d7-72568cd3fd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:10.879586 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f636d-2aa8-44e7-96d7-72568cd3fd7b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:10.884097 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794f4dd-6134-4b8b-813b-4270ea02258a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:10.885029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794f4dd-6134-4b8b-813b-4270ea02258a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:10.885314 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794f4dd-6134-4b8b-813b-4270ea02258a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:10.885585 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794f4dd-6134-4b8b-813b-4270ea02258a admin admin] PATH_INFO: `/v3/users/a3be90df779b4ddb8639d825fcb41234` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:10.886762 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794f4dd-6134-4b8b-813b-4270ea02258a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3be90df779b4ddb8639d825fcb41234)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:10.888805 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794f4dd-6134-4b8b-813b-4270ea02258a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:11.066385 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7794f4dd-6134-4b8b-813b-4270ea02258a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3be90df779b4ddb8639d825fcb41234'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:11.067038 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7794f4dd-6134-4b8b-813b-4270ea02258a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3be90df779b4ddb8639d825fcb41234'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:11.078928 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7794f4dd-6134-4b8b-813b-4270ea02258a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3be90df779b4ddb8639d825fcb41234'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:11.086054 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7794f4dd-6134-4b8b-813b-4270ea02258a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3be90df779b4ddb8639d825fcb41234'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:11.168514 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7794f4dd-6134-4b8b-813b-4270ea02258a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3be90df779b4ddb8639d825fcb41234'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:11.175456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1269/2538] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:14:10 2022] DELETE /identity/v3/users/a3be90df779b4ddb8639d825fcb41234 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:11.396473 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954a86d-a068-4494-b736-daf107fae954 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:11.398092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1954a86d-a068-4494-b736-daf107fae954 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:11.409980 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954a86d-a068-4494-b736-daf107fae954 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:11.423308 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794f4dd-6134-4b8b-813b-4270ea02258a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:11.425000 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7794f4dd-6134-4b8b-813b-4270ea02258a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:11.441516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:11.474883 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36bd33-1836-4f9a-a9aa-18c28d1447b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:11.475533 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36bd33-1836-4f9a-a9aa-18c28d1447b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:11.475848 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36bd33-1836-4f9a-a9aa-18c28d1447b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:11.476117 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36bd33-1836-4f9a-a9aa-18c28d1447b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:11.477926 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ef36bd33-1836-4f9a-a9aa-18c28d1447b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:11.503129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794f4dd-6134-4b8b-813b-4270ea02258a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:11.508001 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9d28c-b720-4035-b9c7-def21a4ba012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:11.509069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9d28c-b720-4035-b9c7-def21a4ba012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:11.509694 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9d28c-b720-4035-b9c7-def21a4ba012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:11.509694 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9d28c-b720-4035-b9c7-def21a4ba012 admin admin] PATH_INFO: `/v3/projects/71c9cca288b34019947e55d2a3112ddb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:11.511301 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9d28c-b720-4035-b9c7-def21a4ba012 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71c9cca288b34019947e55d2a3112ddb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:11.516151 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9d28c-b720-4035-b9c7-def21a4ba012 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:11.519043 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:11.539235 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36bd33-1836-4f9a-a9aa-18c28d1447b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:11.540895 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36bd33-1836-4f9a-a9aa-18c28d1447b5 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:11.550848 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1270/2539] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:14:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:14:11.628004 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8fa9d28c-b720-4035-b9c7-def21a4ba012 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71c9cca288b34019947e55d2a3112ddb', 'request_id': 'req-8fa9d28c-b720-4035-b9c7-def21a4ba012'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:11.634751 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8fa9d28c-b720-4035-b9c7-def21a4ba012 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71c9cca288b34019947e55d2a3112ddb', 'request_id': 'req-8fa9d28c-b720-4035-b9c7-def21a4ba012'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:11.643613 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8fa9d28c-b720-4035-b9c7-def21a4ba012 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71c9cca288b34019947e55d2a3112ddb', 'request_id': 'req-8fa9d28c-b720-4035-b9c7-def21a4ba012'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:11.737468 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8fa9d28c-b720-4035-b9c7-def21a4ba012 admin admin] The token cache is being invalidate because project 71c9cca288b34019947e55d2a3112ddb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:14:11.737756 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8fa9d28c-b720-4035-b9c7-def21a4ba012 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:11.738381 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8fa9d28c-b720-4035-b9c7-def21a4ba012 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:11.739943 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1270/2540] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:14:11 2022] DELETE /identity/v3/projects/71c9cca288b34019947e55d2a3112ddb => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:14:11.745140 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b645e8-e478-4748-ab7d-368974ac9796 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:11.745426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b645e8-e478-4748-ab7d-368974ac9796 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:11.745700 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b645e8-e478-4748-ab7d-368974ac9796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:11.764385 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-28b645e8-e478-4748-ab7d-368974ac9796 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:11.765610 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-28b645e8-e478-4748-ab7d-368974ac9796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:11.823773 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1271/2541] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:11.832434 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6945a-2097-4a99-8493-3af482f4af91 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:11.832925 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6945a-2097-4a99-8493-3af482f4af91 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:11.833061 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6945a-2097-4a99-8493-3af482f4af91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:11.851779 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1ab6945a-2097-4a99-8493-3af482f4af91 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:11.853011 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1ab6945a-2097-4a99-8493-3af482f4af91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:11.860919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1271/2542] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:11.865256 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b645e8-e478-4748-ab7d-368974ac9796 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:11.874822 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b645e8-e478-4748-ab7d-368974ac9796 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:11.879351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192729f1-e043-4bc7-8413-1e55a641b921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:11.880426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192729f1-e043-4bc7-8413-1e55a641b921 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:11.880426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192729f1-e043-4bc7-8413-1e55a641b921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:11.880620 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192729f1-e043-4bc7-8413-1e55a641b921 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:11.881256 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192729f1-e043-4bc7-8413-1e55a641b921 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:14:11.881556 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192729f1-e043-4bc7-8413-1e55a641b921 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:11.883169 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192729f1-e043-4bc7-8413-1e55a641b921 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:11.895159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1272/2543] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 20:14:11.899439 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6945a-2097-4a99-8493-3af482f4af91 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:11.909351 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6945a-2097-4a99-8493-3af482f4af91 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:11.914048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba52d51-3f84-473a-acc0-da10933576bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:11.914714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba52d51-3f84-473a-acc0-da10933576bb admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:11.914994 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba52d51-3f84-473a-acc0-da10933576bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:11.915267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba52d51-3f84-473a-acc0-da10933576bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:11.915973 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba52d51-3f84-473a-acc0-da10933576bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:11.917925 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba52d51-3f84-473a-acc0-da10933576bb admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:12.025333 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1272/2544] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:14:11 2022] POST /identity/v3/projects => generated 381 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:14:12.029627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192729f1-e043-4bc7-8413-1e55a641b921 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:12.039803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192729f1-e043-4bc7-8413-1e55a641b921 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:12.097577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6eecb-aeff-4905-bd3c-8d64dde879f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:12.098158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6eecb-aeff-4905-bd3c-8d64dde879f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:12.098438 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6eecb-aeff-4905-bd3c-8d64dde879f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:12.098725 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6eecb-aeff-4905-bd3c-8d64dde879f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:12.099415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6eecb-aeff-4905-bd3c-8d64dde879f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:12.101374 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6eecb-aeff-4905-bd3c-8d64dde879f7 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:12.212173 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1273/2545] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:14:12 2022] POST /identity/v3/users => generated 291 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:14:12.213681 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba52d51-3f84-473a-acc0-da10933576bb admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:12.224885 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba52d51-3f84-473a-acc0-da10933576bb admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:12.229770 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611bd34-7d81-4c90-9646-352afebf75eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:12.230485 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611bd34-7d81-4c90-9646-352afebf75eb admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:12.230767 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611bd34-7d81-4c90-9646-352afebf75eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:12.231043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611bd34-7d81-4c90-9646-352afebf75eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:12.231772 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0611bd34-7d81-4c90-9646-352afebf75eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:12.233495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0611bd34-7d81-4c90-9646-352afebf75eb admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:12.246883 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1273/2546] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:14:12.251700 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6eecb-aeff-4905-bd3c-8d64dde879f7 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:12.261703 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6eecb-aeff-4905-bd3c-8d64dde879f7 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:12.266890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb285d0c-5a39-47f6-8615-28c2926b65a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:12.267624 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb285d0c-5a39-47f6-8615-28c2926b65a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:12.267908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb285d0c-5a39-47f6-8615-28c2926b65a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:12.268181 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb285d0c-5a39-47f6-8615-28c2926b65a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:12.268941 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb285d0c-5a39-47f6-8615-28c2926b65a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:12.270556 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb285d0c-5a39-47f6-8615-28c2926b65a7 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:12.280612 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1274/2547] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:14:12.284744 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611bd34-7d81-4c90-9646-352afebf75eb admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:12.295126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611bd34-7d81-4c90-9646-352afebf75eb admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:12.299840 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5e576-58c1-4d3e-b5d3-81928f310422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:12.300740 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5e576-58c1-4d3e-b5d3-81928f310422 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:12.301021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5e576-58c1-4d3e-b5d3-81928f310422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:12.301296 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e5e576-58c1-4d3e-b5d3-81928f310422 admin admin] PATH_INFO: `/v3/projects/062cfddbade34875ba395fdd856a1217/users/9425a74235184b019e782bd05f07029c/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:12.316453 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e5e576-58c1-4d3e-b5d3-81928f310422 admin admin] RBAC: Authorizing `identity:create_grant(project_id=062cfddbade34875ba395fdd856a1217, user_id=9425a74235184b019e782bd05f07029c, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:12.324629 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e5e576-58c1-4d3e-b5d3-81928f310422 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:12.433141 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1274/2548] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:14:12 2022] PUT /identity/v3/projects/062cfddbade34875ba395fdd856a1217/users/9425a74235184b019e782bd05f07029c/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:12.444497 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee7aa6-1474-4bb0-891e-cc2411d214b5 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:12.445013 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee7aa6-1474-4bb0-891e-cc2411d214b5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:12.445104 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee7aa6-1474-4bb0-891e-cc2411d214b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:12.471123 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f3ee7aa6-1474-4bb0-891e-cc2411d214b5 None None] MFA Rules not processed for user `9425a74235184b019e782bd05f07029c`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:12.472362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f3ee7aa6-1474-4bb0-891e-cc2411d214b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:12.504619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:12.535632 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1275/2549] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:12.540177 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5e576-58c1-4d3e-b5d3-81928f310422 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:12.574868 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:12.602740 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e5e576-58c1-4d3e-b5d3-81928f310422 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:12.608041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c9d87-dbf9-4f7c-8ef7-dc24d4c560e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:12.610671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c9d87-dbf9-4f7c-8ef7-dc24d4c560e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:12.610671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c9d87-dbf9-4f7c-8ef7-dc24d4c560e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:12.610671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c9d87-dbf9-4f7c-8ef7-dc24d4c560e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:12.610671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c9d87-dbf9-4f7c-8ef7-dc24d4c560e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:12.611972 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c9d87-dbf9-4f7c-8ef7-dc24d4c560e5 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:12.728167 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1275/2550] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:14:12 2022] POST /identity/v3/projects => generated 379 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:14:12.732251 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee7aa6-1474-4bb0-891e-cc2411d214b5 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:12.743565 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee7aa6-1474-4bb0-891e-cc2411d214b5 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:12.801393 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe2e5e-f7a3-4114-b86a-1bfa5e3cf9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:12.802084 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe2e5e-f7a3-4114-b86a-1bfa5e3cf9f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:12.802367 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe2e5e-f7a3-4114-b86a-1bfa5e3cf9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:12.802747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe2e5e-f7a3-4114-b86a-1bfa5e3cf9f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:12.803430 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe2e5e-f7a3-4114-b86a-1bfa5e3cf9f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:12.805422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe2e5e-f7a3-4114-b86a-1bfa5e3cf9f3 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:12.900388 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1276/2551] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:14:12 2022] POST /identity/v3/users => generated 296 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:14:12.904470 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c9d87-dbf9-4f7c-8ef7-dc24d4c560e5 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:12.915242 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c9d87-dbf9-4f7c-8ef7-dc24d4c560e5 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:12.919905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a561024a-0573-46d4-ae20-ba009f38da50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:12.921393 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561024a-0573-46d4-ae20-ba009f38da50 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:12.921848 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561024a-0573-46d4-ae20-ba009f38da50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:12.922568 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a561024a-0573-46d4-ae20-ba009f38da50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:12.923414 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a561024a-0573-46d4-ae20-ba009f38da50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:12.925149 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a561024a-0573-46d4-ae20-ba009f38da50 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:12.936639 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1276/2552] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:14:12.946025 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe2e5e-f7a3-4114-b86a-1bfa5e3cf9f3 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:12.956191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe2e5e-f7a3-4114-b86a-1bfa5e3cf9f3 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:12.961265 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450d660-db29-4b9a-90b2-34e4f60b7438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:12.962144 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450d660-db29-4b9a-90b2-34e4f60b7438 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:12.962427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450d660-db29-4b9a-90b2-34e4f60b7438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:12.962707 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450d660-db29-4b9a-90b2-34e4f60b7438 admin admin] PATH_INFO: `/v3/projects/baace296fc1945d49be40bb9de8e8bba/users/d163475db99148e7aabe2b5a225be2fa/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:12.978360 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d450d660-db29-4b9a-90b2-34e4f60b7438 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baace296fc1945d49be40bb9de8e8bba, user_id=d163475db99148e7aabe2b5a225be2fa, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:12.981034 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d450d660-db29-4b9a-90b2-34e4f60b7438 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:13.073623 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1277/2553] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:14:12 2022] PUT /identity/v3/projects/baace296fc1945d49be40bb9de8e8bba/users/d163475db99148e7aabe2b5a225be2fa/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:13.077905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a561024a-0573-46d4-ae20-ba009f38da50 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:13.091639 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a561024a-0573-46d4-ae20-ba009f38da50 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:13.100802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450d660-db29-4b9a-90b2-34e4f60b7438 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:13.102535 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d450d660-db29-4b9a-90b2-34e4f60b7438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:13.149160 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b9667e-e855-474e-a1b4-50ff0e72ad29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:13.149829 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9667e-e855-474e-a1b4-50ff0e72ad29 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:13.150114 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9667e-e855-474e-a1b4-50ff0e72ad29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:13.150386 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9667e-e855-474e-a1b4-50ff0e72ad29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:13.151136 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b9667e-e855-474e-a1b4-50ff0e72ad29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:13.152750 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b9667e-e855-474e-a1b4-50ff0e72ad29 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:13.160204 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450d660-db29-4b9a-90b2-34e4f60b7438 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:13.166474 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1277/2554] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:14:13.166623 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed579f2-40db-400b-98e6-7485ff394c45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:13.166623 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed579f2-40db-400b-98e6-7485ff394c45 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:13.166623 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed579f2-40db-400b-98e6-7485ff394c45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:13.166623 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed579f2-40db-400b-98e6-7485ff394c45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:13.167011 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bed579f2-40db-400b-98e6-7485ff394c45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:13.167472 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b9667e-e855-474e-a1b4-50ff0e72ad29 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:13.177094 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b9667e-e855-474e-a1b4-50ff0e72ad29 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:13.181836 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b2126-9e0c-4934-8d20-2cd4ae2e230f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:13.182725 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b2126-9e0c-4934-8d20-2cd4ae2e230f admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:13.183009 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b2126-9e0c-4934-8d20-2cd4ae2e230f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:13.183284 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b2126-9e0c-4934-8d20-2cd4ae2e230f admin admin] PATH_INFO: `/v3/projects/baace296fc1945d49be40bb9de8e8bba/users/d163475db99148e7aabe2b5a225be2fa/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:13.185281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b2126-9e0c-4934-8d20-2cd4ae2e230f admin admin] RBAC: Authorizing `identity:create_grant(project_id=baace296fc1945d49be40bb9de8e8bba, user_id=d163475db99148e7aabe2b5a225be2fa, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:13.188021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b2126-9e0c-4934-8d20-2cd4ae2e230f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:13.198544 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1278/2555] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:14:13 2022] PUT /identity/v3/projects/baace296fc1945d49be40bb9de8e8bba/users/d163475db99148e7aabe2b5a225be2fa/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:13.209386 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730fbbc-9e8f-47c7-966b-af8c7e1839c0 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:13.209679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730fbbc-9e8f-47c7-966b-af8c7e1839c0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:13.212729 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730fbbc-9e8f-47c7-966b-af8c7e1839c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:13.219565 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:13.233214 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed579f2-40db-400b-98e6-7485ff394c45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:13.234605 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed579f2-40db-400b-98e6-7485ff394c45 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:13.240387 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9730fbbc-9e8f-47c7-966b-af8c7e1839c0 None None] MFA Rules not processed for user `d163475db99148e7aabe2b5a225be2fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:13.246477 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9730fbbc-9e8f-47c7-966b-af8c7e1839c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:13.246601 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1278/2556] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:14:13 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 20:14:13.314075 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1279/2557] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:13 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:14:13.325354 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed579f2-40db-400b-98e6-7485ff394c45 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:13.327283 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bed579f2-40db-400b-98e6-7485ff394c45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:13.338346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed579f2-40db-400b-98e6-7485ff394c45 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:13.387399 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c5785-2b19-49e0-810f-8e10f5550d6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:13.388093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c5785-2b19-49e0-810f-8e10f5550d6b service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:13.388410 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c5785-2b19-49e0-810f-8e10f5550d6b service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:13.388688 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5c5785-2b19-49e0-810f-8e10f5550d6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:13.398392 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5c5785-2b19-49e0-810f-8e10f5550d6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:13.399824 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5c5785-2b19-49e0-810f-8e10f5550d6b service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:13.409847 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1279/2558] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:14:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:14:13.626768 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730fbbc-9e8f-47c7-966b-af8c7e1839c0 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:13.636514 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730fbbc-9e8f-47c7-966b-af8c7e1839c0 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:13.640615 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ba69e-1e16-416a-86ec-babfe212fcb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:13.641253 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ba69e-1e16-416a-86ec-babfe212fcb8 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:13.641537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ba69e-1e16-416a-86ec-babfe212fcb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:13.641811 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ba69e-1e16-416a-86ec-babfe212fcb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:13.651067 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:13.699110 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ba69e-1e16-416a-86ec-babfe212fcb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:13.700712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ba69e-1e16-416a-86ec-babfe212fcb8 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:13.710549 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1280/2559] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:14:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:14:17.912813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c5785-2b19-49e0-810f-8e10f5550d6b service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:17.914195 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ce5c5785-2b19-49e0-810f-8e10f5550d6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:17.977739 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5c5785-2b19-49e0-810f-8e10f5550d6b service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:17.982444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea1584-5487-4003-81b3-c89d30f4ad70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:17.983463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea1584-5487-4003-81b3-c89d30f4ad70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:17.983744 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea1584-5487-4003-81b3-c89d30f4ad70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:17.984015 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea1584-5487-4003-81b3-c89d30f4ad70 admin admin] PATH_INFO: `/v3/users/f42eaa94d7a547139b51503b24937ff7` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:17.985458 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea1584-5487-4003-81b3-c89d30f4ad70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f42eaa94d7a547139b51503b24937ff7)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:17.987412 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea1584-5487-4003-81b3-c89d30f4ad70 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:18.173439 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-52ea1584-5487-4003-81b3-c89d30f4ad70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f42eaa94d7a547139b51503b24937ff7'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:18.174090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-52ea1584-5487-4003-81b3-c89d30f4ad70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f42eaa94d7a547139b51503b24937ff7'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:18.466271 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-52ea1584-5487-4003-81b3-c89d30f4ad70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f42eaa94d7a547139b51503b24937ff7'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:18.466412 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-52ea1584-5487-4003-81b3-c89d30f4ad70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f42eaa94d7a547139b51503b24937ff7'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:18.480000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-52ea1584-5487-4003-81b3-c89d30f4ad70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f42eaa94d7a547139b51503b24937ff7'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:18.493480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1280/2560] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:14:17 2022] DELETE /identity/v3/users/f42eaa94d7a547139b51503b24937ff7 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:18.497957 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c923b-e828-4a21-a1e0-52999570d024 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:18.499082 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c923b-e828-4a21-a1e0-52999570d024 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:18.499211 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c923b-e828-4a21-a1e0-52999570d024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:18.516729 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-185c923b-e828-4a21-a1e0-52999570d024 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:18.517928 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-185c923b-e828-4a21-a1e0-52999570d024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:18.575358 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1281/2561] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:18.585187 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea1584-5487-4003-81b3-c89d30f4ad70 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:18.586630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-52ea1584-5487-4003-81b3-c89d30f4ad70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:18.599960 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:18.643154 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea1584-5487-4003-81b3-c89d30f4ad70 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:18.646756 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b4d47-1cb7-48bd-bff8-625e11b8f20a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:18.647392 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b4d47-1cb7-48bd-bff8-625e11b8f20a service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:18.647670 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b4d47-1cb7-48bd-bff8-625e11b8f20a service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:18.647942 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b4d47-1cb7-48bd-bff8-625e11b8f20a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:18.656714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b4d47-1cb7-48bd-bff8-625e11b8f20a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:18.658142 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b4d47-1cb7-48bd-bff8-625e11b8f20a service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:18.667432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1281/2562] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:14:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:14:19.330259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c923b-e828-4a21-a1e0-52999570d024 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:19.332048 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-185c923b-e828-4a21-a1e0-52999570d024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:19.345165 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185c923b-e828-4a21-a1e0-52999570d024 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:19.349853 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f0545-6952-4a38-9dfc-793f2ece605a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:19.350710 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f0545-6952-4a38-9dfc-793f2ece605a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:19.350987 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f0545-6952-4a38-9dfc-793f2ece605a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:19.351256 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f0545-6952-4a38-9dfc-793f2ece605a admin admin] PATH_INFO: `/v3/projects/2c88db35d2f747fbbedc5b2dc4fecbc2` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:19.352479 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f0545-6952-4a38-9dfc-793f2ece605a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c88db35d2f747fbbedc5b2dc4fecbc2)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:19.354448 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f0545-6952-4a38-9dfc-793f2ece605a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:19.488705 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fa2f0545-6952-4a38-9dfc-793f2ece605a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c88db35d2f747fbbedc5b2dc4fecbc2', 'request_id': 'req-fa2f0545-6952-4a38-9dfc-793f2ece605a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:19.495727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fa2f0545-6952-4a38-9dfc-793f2ece605a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c88db35d2f747fbbedc5b2dc4fecbc2', 'request_id': 'req-fa2f0545-6952-4a38-9dfc-793f2ece605a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:19.502494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fa2f0545-6952-4a38-9dfc-793f2ece605a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c88db35d2f747fbbedc5b2dc4fecbc2', 'request_id': 'req-fa2f0545-6952-4a38-9dfc-793f2ece605a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:19.562448 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fa2f0545-6952-4a38-9dfc-793f2ece605a admin admin] The token cache is being invalidate because project 2c88db35d2f747fbbedc5b2dc4fecbc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:14:19.562729 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fa2f0545-6952-4a38-9dfc-793f2ece605a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:19.563390 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fa2f0545-6952-4a38-9dfc-793f2ece605a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:19.565041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1282/2563] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:14:19 2022] DELETE /identity/v3/projects/2c88db35d2f747fbbedc5b2dc4fecbc2 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:19.569462 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b4d47-1cb7-48bd-bff8-625e11b8f20a service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:19.579625 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b4d47-1cb7-48bd-bff8-625e11b8f20a service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:19.634606 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d6b89-5ef4-446e-a0d4-a297df31647e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:19.635411 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d6b89-5ef4-446e-a0d4-a297df31647e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:19.635699 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d6b89-5ef4-446e-a0d4-a297df31647e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:19.635974 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d6b89-5ef4-446e-a0d4-a297df31647e admin admin] PATH_INFO: `/v3/users/8a8ff70a673340f5bd2d942ff633e5b2` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:19.637245 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d6b89-5ef4-446e-a0d4-a297df31647e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a8ff70a673340f5bd2d942ff633e5b2)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:19.639875 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d6b89-5ef4-446e-a0d4-a297df31647e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:19.814420 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-111d6b89-5ef4-446e-a0d4-a297df31647e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a8ff70a673340f5bd2d942ff633e5b2'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:19.815094 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-111d6b89-5ef4-446e-a0d4-a297df31647e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a8ff70a673340f5bd2d942ff633e5b2'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:19.960792 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-111d6b89-5ef4-446e-a0d4-a297df31647e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a8ff70a673340f5bd2d942ff633e5b2'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:19.961346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-111d6b89-5ef4-446e-a0d4-a297df31647e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a8ff70a673340f5bd2d942ff633e5b2'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:19.973916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-111d6b89-5ef4-446e-a0d4-a297df31647e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a8ff70a673340f5bd2d942ff633e5b2'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:19.986503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1282/2564] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:14:19 2022] DELETE /identity/v3/users/8a8ff70a673340f5bd2d942ff633e5b2 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:20.426518 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f0545-6952-4a38-9dfc-793f2ece605a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:20.428036 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fa2f0545-6952-4a38-9dfc-793f2ece605a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:20.441803 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f0545-6952-4a38-9dfc-793f2ece605a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:20.475516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:20.499370 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ceb85e-0d30-4d84-9560-27302dc4bc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:20.500229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ceb85e-0d30-4d84-9560-27302dc4bc34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:20.500540 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ceb85e-0d30-4d84-9560-27302dc4bc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:20.500814 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ceb85e-0d30-4d84-9560-27302dc4bc34 admin admin] PATH_INFO: `/v3/projects/d8ae82748b114685a74497f005b9ca54` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:20.501988 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ceb85e-0d30-4d84-9560-27302dc4bc34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8ae82748b114685a74497f005b9ca54)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:20.504195 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ceb85e-0d30-4d84-9560-27302dc4bc34 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:20.639576 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-40ceb85e-0d30-4d84-9560-27302dc4bc34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8ae82748b114685a74497f005b9ca54', 'request_id': 'req-40ceb85e-0d30-4d84-9560-27302dc4bc34'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:20.648336 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-40ceb85e-0d30-4d84-9560-27302dc4bc34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8ae82748b114685a74497f005b9ca54', 'request_id': 'req-40ceb85e-0d30-4d84-9560-27302dc4bc34'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:20.655223 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-40ceb85e-0d30-4d84-9560-27302dc4bc34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8ae82748b114685a74497f005b9ca54', 'request_id': 'req-40ceb85e-0d30-4d84-9560-27302dc4bc34'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:20.744907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-40ceb85e-0d30-4d84-9560-27302dc4bc34 admin admin] The token cache is being invalidate because project d8ae82748b114685a74497f005b9ca54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:14:20.750504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-40ceb85e-0d30-4d84-9560-27302dc4bc34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:20.750504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-40ceb85e-0d30-4d84-9560-27302dc4bc34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:20.750504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1283/2565] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:14:20 2022] DELETE /identity/v3/projects/d8ae82748b114685a74497f005b9ca54 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:20.752057 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d6b89-5ef4-446e-a0d4-a297df31647e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:20.753486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-111d6b89-5ef4-446e-a0d4-a297df31647e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:20.766162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d6b89-5ef4-446e-a0d4-a297df31647e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:20.796534 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:20.826527 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658eb3b-aae3-45f9-9f36-2015669d874c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:20.827408 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658eb3b-aae3-45f9-9f36-2015669d874c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:20.827691 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658eb3b-aae3-45f9-9f36-2015669d874c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:20.827965 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658eb3b-aae3-45f9-9f36-2015669d874c admin admin] PATH_INFO: `/v3/users/eb5b6f1a015349389e8ae9591da174ff` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:20.829193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658eb3b-aae3-45f9-9f36-2015669d874c admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb5b6f1a015349389e8ae9591da174ff)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:20.831138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658eb3b-aae3-45f9-9f36-2015669d874c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:21.031213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2658eb3b-aae3-45f9-9f36-2015669d874c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb5b6f1a015349389e8ae9591da174ff'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:21.031815 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2658eb3b-aae3-45f9-9f36-2015669d874c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb5b6f1a015349389e8ae9591da174ff'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:21.103245 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2658eb3b-aae3-45f9-9f36-2015669d874c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb5b6f1a015349389e8ae9591da174ff'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:21.103848 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2658eb3b-aae3-45f9-9f36-2015669d874c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb5b6f1a015349389e8ae9591da174ff'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:21.116200 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2658eb3b-aae3-45f9-9f36-2015669d874c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb5b6f1a015349389e8ae9591da174ff'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:21.128429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1283/2566] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:14:20 2022] DELETE /identity/v3/users/eb5b6f1a015349389e8ae9591da174ff => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:21.536494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ceb85e-0d30-4d84-9560-27302dc4bc34 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:21.537983 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-40ceb85e-0d30-4d84-9560-27302dc4bc34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:21.554891 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ceb85e-0d30-4d84-9560-27302dc4bc34 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:21.622744 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd350653-7e29-4d78-a4bf-81f062860112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:21.623630 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd350653-7e29-4d78-a4bf-81f062860112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:21.624015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd350653-7e29-4d78-a4bf-81f062860112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:21.624290 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd350653-7e29-4d78-a4bf-81f062860112 admin admin] PATH_INFO: `/v3/projects/78f7863c23584d4aaba9046ec14d5c1a` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:21.625497 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd350653-7e29-4d78-a4bf-81f062860112 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78f7863c23584d4aaba9046ec14d5c1a)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:21.627524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd350653-7e29-4d78-a4bf-81f062860112 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:21.757680 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fd350653-7e29-4d78-a4bf-81f062860112 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78f7863c23584d4aaba9046ec14d5c1a', 'request_id': 'req-fd350653-7e29-4d78-a4bf-81f062860112'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:21.765035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fd350653-7e29-4d78-a4bf-81f062860112 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78f7863c23584d4aaba9046ec14d5c1a', 'request_id': 'req-fd350653-7e29-4d78-a4bf-81f062860112'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:21.770507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fd350653-7e29-4d78-a4bf-81f062860112 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78f7863c23584d4aaba9046ec14d5c1a', 'request_id': 'req-fd350653-7e29-4d78-a4bf-81f062860112'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:21.774157 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:21.859984 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fd350653-7e29-4d78-a4bf-81f062860112 admin admin] The token cache is being invalidate because project 78f7863c23584d4aaba9046ec14d5c1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:14:21.860271 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fd350653-7e29-4d78-a4bf-81f062860112 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:21.860930 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fd350653-7e29-4d78-a4bf-81f062860112 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:21.862467 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1284/2567] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:14:21 2022] DELETE /identity/v3/projects/78f7863c23584d4aaba9046ec14d5c1a => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:21.867372 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba478aba-a64f-4558-8101-aa97bd31ff41 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:21.867831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba478aba-a64f-4558-8101-aa97bd31ff41 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:21.867923 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba478aba-a64f-4558-8101-aa97bd31ff41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:21.887005 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ba478aba-a64f-4558-8101-aa97bd31ff41 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:21.888211 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ba478aba-a64f-4558-8101-aa97bd31ff41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:21.947100 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1284/2568] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:21.955689 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9208a-2b61-4fb0-a812-5bc9fc3eeab2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:21.955977 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9208a-2b61-4fb0-a812-5bc9fc3eeab2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:21.956256 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9208a-2b61-4fb0-a812-5bc9fc3eeab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:21.974943 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d1c9208a-2b61-4fb0-a812-5bc9fc3eeab2 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:21.979035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d1c9208a-2b61-4fb0-a812-5bc9fc3eeab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:21.984279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1285/2569] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:21.988621 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba478aba-a64f-4558-8101-aa97bd31ff41 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:21.999409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba478aba-a64f-4558-8101-aa97bd31ff41 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:22.004413 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac932db2-4638-4676-8aa8-45a04ab0200c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:22.009541 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac932db2-4638-4676-8aa8-45a04ab0200c admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:22.009541 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac932db2-4638-4676-8aa8-45a04ab0200c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:22.009541 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac932db2-4638-4676-8aa8-45a04ab0200c admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:22.009541 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac932db2-4638-4676-8aa8-45a04ab0200c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:14:22.009541 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac932db2-4638-4676-8aa8-45a04ab0200c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:22.009541 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac932db2-4638-4676-8aa8-45a04ab0200c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:22.023575 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1285/2570] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:14:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:14:22.029236 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9208a-2b61-4fb0-a812-5bc9fc3eeab2 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:22.040626 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9208a-2b61-4fb0-a812-5bc9fc3eeab2 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:22.047027 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb5186-7680-4ca9-b299-253e9aa02293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:22.047843 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb5186-7680-4ca9-b299-253e9aa02293 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:22.048129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb5186-7680-4ca9-b299-253e9aa02293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:22.048507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb5186-7680-4ca9-b299-253e9aa02293 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:22.049234 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb5186-7680-4ca9-b299-253e9aa02293 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:22.051294 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb5186-7680-4ca9-b299-253e9aa02293 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:22.117931 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1286/2571] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:14:22 2022] POST /identity/v3/projects => generated 369 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:14:22.118967 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac932db2-4638-4676-8aa8-45a04ab0200c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:22.129759 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac932db2-4638-4676-8aa8-45a04ab0200c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:22.168167 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:22.186830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f68840-37c0-4a94-8ff8-c602b720e35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:22.187499 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f68840-37c0-4a94-8ff8-c602b720e35d admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:22.187777 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f68840-37c0-4a94-8ff8-c602b720e35d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:22.188050 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f68840-37c0-4a94-8ff8-c602b720e35d admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:22.188772 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f68840-37c0-4a94-8ff8-c602b720e35d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:22.190727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f68840-37c0-4a94-8ff8-c602b720e35d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:22.316180 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1286/2572] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:14:22 2022] POST /identity/v3/users => generated 285 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:14:22.321494 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb5186-7680-4ca9-b299-253e9aa02293 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:22.331942 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb5186-7680-4ca9-b299-253e9aa02293 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:22.336663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b7661-f01a-416a-b677-e36237700606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:22.337319 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b7661-f01a-416a-b677-e36237700606 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:22.337597 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b7661-f01a-416a-b677-e36237700606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:22.337866 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b7661-f01a-416a-b677-e36237700606 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:22.338627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b7661-f01a-416a-b677-e36237700606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:22.340233 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b7661-f01a-416a-b677-e36237700606 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:22.351529 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1287/2573] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:14:22.355564 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f68840-37c0-4a94-8ff8-c602b720e35d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:22.366415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f68840-37c0-4a94-8ff8-c602b720e35d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:22.371125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a689280-9290-47f4-8929-652339696272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:22.371772 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a689280-9290-47f4-8929-652339696272 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:22.372050 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a689280-9290-47f4-8929-652339696272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:22.372351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a689280-9290-47f4-8929-652339696272 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:22.373075 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a689280-9290-47f4-8929-652339696272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:22.375317 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a689280-9290-47f4-8929-652339696272 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:22.390342 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1287/2574] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:14:22.391359 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b7661-f01a-416a-b677-e36237700606 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:22.402886 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b7661-f01a-416a-b677-e36237700606 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:22.408089 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1815df7-9b82-4b6d-8ad3-4999e3f05358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:22.409038 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1815df7-9b82-4b6d-8ad3-4999e3f05358 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:22.409318 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1815df7-9b82-4b6d-8ad3-4999e3f05358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:22.409602 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1815df7-9b82-4b6d-8ad3-4999e3f05358 admin admin] PATH_INFO: `/v3/projects/cf33365cc53d4914b29ad7448f8edd4a/users/7b559ee39b7d4624b81791e9f66e9e72/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:22.426937 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1815df7-9b82-4b6d-8ad3-4999e3f05358 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf33365cc53d4914b29ad7448f8edd4a, user_id=7b559ee39b7d4624b81791e9f66e9e72, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:22.429641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1815df7-9b82-4b6d-8ad3-4999e3f05358 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:22.517278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1288/2575] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:14:22 2022] PUT /identity/v3/projects/cf33365cc53d4914b29ad7448f8edd4a/users/7b559ee39b7d4624b81791e9f66e9e72/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:22.528209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55253433-c556-4187-aeeb-7514f31dec1f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:22.528529 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55253433-c556-4187-aeeb-7514f31dec1f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:22.528826 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55253433-c556-4187-aeeb-7514f31dec1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:22.555512 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-55253433-c556-4187-aeeb-7514f31dec1f None None] MFA Rules not processed for user `7b559ee39b7d4624b81791e9f66e9e72`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:22.556757 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-55253433-c556-4187-aeeb-7514f31dec1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:22.620117 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1288/2576] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:22 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:22.632081 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1815df7-9b82-4b6d-8ad3-4999e3f05358 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:22.633794 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b1815df7-9b82-4b6d-8ad3-4999e3f05358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:22.689135 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1815df7-9b82-4b6d-8ad3-4999e3f05358 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:22.692776 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45f7954-4b67-4e13-b408-439194fc8134 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd72f952df34ad8853fe84673a41f45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:22.693412 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f7954-4b67-4e13-b408-439194fc8134 service glance] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:22.693691 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f7954-4b67-4e13-b408-439194fc8134 service glance] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:22.693962 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f7954-4b67-4e13-b408-439194fc8134 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:22.702091 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45f7954-4b67-4e13-b408-439194fc8134 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:22.703512 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45f7954-4b67-4e13-b408-439194fc8134 service glance] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:22.712735 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1289/2577] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:14:22 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:14:25.221123 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55253433-c556-4187-aeeb-7514f31dec1f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:25.222532 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-55253433-c556-4187-aeeb-7514f31dec1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:25.297176 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55253433-c556-4187-aeeb-7514f31dec1f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:25.301680 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2afeea-c610-4020-9ece-05388528bdbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:25.302577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2afeea-c610-4020-9ece-05388528bdbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:25.302859 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2afeea-c610-4020-9ece-05388528bdbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:25.303133 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2afeea-c610-4020-9ece-05388528bdbf admin admin] PATH_INFO: `/v3/users/9425a74235184b019e782bd05f07029c` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:25.304343 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2afeea-c610-4020-9ece-05388528bdbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=9425a74235184b019e782bd05f07029c)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:25.308138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2afeea-c610-4020-9ece-05388528bdbf admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:25.557881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-bc2afeea-c610-4020-9ece-05388528bdbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9425a74235184b019e782bd05f07029c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:25.558470 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-bc2afeea-c610-4020-9ece-05388528bdbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9425a74235184b019e782bd05f07029c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:25.560969 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:25.665338 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-bc2afeea-c610-4020-9ece-05388528bdbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9425a74235184b019e782bd05f07029c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:25.666101 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-bc2afeea-c610-4020-9ece-05388528bdbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9425a74235184b019e782bd05f07029c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:25.678755 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-bc2afeea-c610-4020-9ece-05388528bdbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9425a74235184b019e782bd05f07029c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:25.691501 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1289/2578] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:14:25 2022] DELETE /identity/v3/users/9425a74235184b019e782bd05f07029c => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:25.702699 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee7486-3043-4c9d-b20d-28cf671315c9 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:25.703231 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee7486-3043-4c9d-b20d-28cf671315c9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:25.703231 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee7486-3043-4c9d-b20d-28cf671315c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:25.722580 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-2cee7486-3043-4c9d-b20d-28cf671315c9 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:25.723896 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2cee7486-3043-4c9d-b20d-28cf671315c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:25.784188 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1290/2579] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:25.793842 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2afeea-c610-4020-9ece-05388528bdbf admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:25.795238 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bc2afeea-c610-4020-9ece-05388528bdbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:25.852932 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2afeea-c610-4020-9ece-05388528bdbf admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:25.856536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f1274-c73e-4e33-bd97-d181eb59926a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:25.857180 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f1274-c73e-4e33-bd97-d181eb59926a service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:25.857461 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f1274-c73e-4e33-bd97-d181eb59926a service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:25.857733 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f1274-c73e-4e33-bd97-d181eb59926a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:25.866499 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f1274-c73e-4e33-bd97-d181eb59926a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:25.867968 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f1274-c73e-4e33-bd97-d181eb59926a service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:25.877388 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1290/2580] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:14:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:14:26.291947 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee7486-3043-4c9d-b20d-28cf671315c9 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:26.293492 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2cee7486-3043-4c9d-b20d-28cf671315c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:26.305705 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee7486-3043-4c9d-b20d-28cf671315c9 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:26.310548 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eefa30-844d-45c4-b320-a8f00cad8eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:26.311426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eefa30-844d-45c4-b320-a8f00cad8eec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:26.312215 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eefa30-844d-45c4-b320-a8f00cad8eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:26.312502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eefa30-844d-45c4-b320-a8f00cad8eec admin admin] PATH_INFO: `/v3/projects/062cfddbade34875ba395fdd856a1217` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:26.313698 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eefa30-844d-45c4-b320-a8f00cad8eec admin admin] RBAC: Authorizing `identity:delete_project(project_id=062cfddbade34875ba395fdd856a1217)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:26.315697 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eefa30-844d-45c4-b320-a8f00cad8eec admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:26.419750 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:26.433924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e7eefa30-844d-45c4-b320-a8f00cad8eec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '062cfddbade34875ba395fdd856a1217', 'request_id': 'req-e7eefa30-844d-45c4-b320-a8f00cad8eec'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:26.440625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e7eefa30-844d-45c4-b320-a8f00cad8eec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '062cfddbade34875ba395fdd856a1217', 'request_id': 'req-e7eefa30-844d-45c4-b320-a8f00cad8eec'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:26.446056 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e7eefa30-844d-45c4-b320-a8f00cad8eec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '062cfddbade34875ba395fdd856a1217', 'request_id': 'req-e7eefa30-844d-45c4-b320-a8f00cad8eec'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:26.549891 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e7eefa30-844d-45c4-b320-a8f00cad8eec admin admin] The token cache is being invalidate because project 062cfddbade34875ba395fdd856a1217 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:14:26.550181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e7eefa30-844d-45c4-b320-a8f00cad8eec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:26.550841 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e7eefa30-844d-45c4-b320-a8f00cad8eec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:26.552424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1291/2581] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:14:26 2022] DELETE /identity/v3/projects/062cfddbade34875ba395fdd856a1217 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:26.556658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f1274-c73e-4e33-bd97-d181eb59926a service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:26.567490 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f1274-c73e-4e33-bd97-d181eb59926a service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:26.629175 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0017fd0-cea1-49e0-a47f-f07195344ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:26.630206 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0017fd0-cea1-49e0-a47f-f07195344ff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:26.631539 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0017fd0-cea1-49e0-a47f-f07195344ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:26.631825 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0017fd0-cea1-49e0-a47f-f07195344ff2 admin admin] PATH_INFO: `/v3/users/d163475db99148e7aabe2b5a225be2fa` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:26.633198 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0017fd0-cea1-49e0-a47f-f07195344ff2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d163475db99148e7aabe2b5a225be2fa)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:26.635071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0017fd0-cea1-49e0-a47f-f07195344ff2 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:26.845486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c0017fd0-cea1-49e0-a47f-f07195344ff2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd163475db99148e7aabe2b5a225be2fa'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:26.847107 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c0017fd0-cea1-49e0-a47f-f07195344ff2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd163475db99148e7aabe2b5a225be2fa'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:26.923184 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c0017fd0-cea1-49e0-a47f-f07195344ff2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd163475db99148e7aabe2b5a225be2fa'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:26.923810 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c0017fd0-cea1-49e0-a47f-f07195344ff2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd163475db99148e7aabe2b5a225be2fa'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:26.939694 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c0017fd0-cea1-49e0-a47f-f07195344ff2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd163475db99148e7aabe2b5a225be2fa'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:26.943237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:26.953659 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1291/2582] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:14:26 2022] DELETE /identity/v3/users/d163475db99148e7aabe2b5a225be2fa => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:27.419180 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eefa30-844d-45c4-b320-a8f00cad8eec admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:27.423628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e7eefa30-844d-45c4-b320-a8f00cad8eec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:27.436317 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eefa30-844d-45c4-b320-a8f00cad8eec admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:27.495023 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f95502-bfc4-4d28-8cf0-71ee63ed09c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:27.495888 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f95502-bfc4-4d28-8cf0-71ee63ed09c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:27.496190 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f95502-bfc4-4d28-8cf0-71ee63ed09c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:27.496490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f95502-bfc4-4d28-8cf0-71ee63ed09c4 admin admin] PATH_INFO: `/v3/projects/baace296fc1945d49be40bb9de8e8bba` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:27.497732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f95502-bfc4-4d28-8cf0-71ee63ed09c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=baace296fc1945d49be40bb9de8e8bba)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:27.500473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f95502-bfc4-4d28-8cf0-71ee63ed09c4 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:27.630399 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a0f95502-bfc4-4d28-8cf0-71ee63ed09c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baace296fc1945d49be40bb9de8e8bba', 'request_id': 'req-a0f95502-bfc4-4d28-8cf0-71ee63ed09c4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:27.636552 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a0f95502-bfc4-4d28-8cf0-71ee63ed09c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baace296fc1945d49be40bb9de8e8bba', 'request_id': 'req-a0f95502-bfc4-4d28-8cf0-71ee63ed09c4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:27.642197 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a0f95502-bfc4-4d28-8cf0-71ee63ed09c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baace296fc1945d49be40bb9de8e8bba', 'request_id': 'req-a0f95502-bfc4-4d28-8cf0-71ee63ed09c4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:27.731756 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a0f95502-bfc4-4d28-8cf0-71ee63ed09c4 admin admin] The token cache is being invalidate because project baace296fc1945d49be40bb9de8e8bba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:14:27.736882 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a0f95502-bfc4-4d28-8cf0-71ee63ed09c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:27.736882 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a0f95502-bfc4-4d28-8cf0-71ee63ed09c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:27.736882 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1292/2583] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:14:27 2022] DELETE /identity/v3/projects/baace296fc1945d49be40bb9de8e8bba => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:27.740428 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf903c-a131-4abc-91ce-65913909c529 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:27.740869 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf903c-a131-4abc-91ce-65913909c529 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:27.740988 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf903c-a131-4abc-91ce-65913909c529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:27.760838 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-93cf903c-a131-4abc-91ce-65913909c529 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:27.762065 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-93cf903c-a131-4abc-91ce-65913909c529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:27.825060 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1292/2584] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:27.835469 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96233abe-d69c-4a12-913b-00893fb95f7d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:27.835753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96233abe-d69c-4a12-913b-00893fb95f7d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:27.836029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96233abe-d69c-4a12-913b-00893fb95f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:27.855043 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-96233abe-d69c-4a12-913b-00893fb95f7d None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:27.856417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-96233abe-d69c-4a12-913b-00893fb95f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:27.865654 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1293/2585] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:27.870381 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf903c-a131-4abc-91ce-65913909c529 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:27.880022 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf903c-a131-4abc-91ce-65913909c529 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:27.884482 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a2e06-5dd5-4719-bb4c-07902337ca3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:27.885120 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a2e06-5dd5-4719-bb4c-07902337ca3c admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:27.885400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a2e06-5dd5-4719-bb4c-07902337ca3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:27.885671 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a2e06-5dd5-4719-bb4c-07902337ca3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:27.886385 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a2e06-5dd5-4719-bb4c-07902337ca3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:14:27.886685 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a2e06-5dd5-4719-bb4c-07902337ca3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:27.888284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a2e06-5dd5-4719-bb4c-07902337ca3c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:27.901862 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1293/2586] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:14:27.906572 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96233abe-d69c-4a12-913b-00893fb95f7d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:27.915943 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96233abe-d69c-4a12-913b-00893fb95f7d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:27.920854 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482ff85-c759-4dcf-a0ec-d8880dd9ccda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:27.921507 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482ff85-c759-4dcf-a0ec-d8880dd9ccda admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:27.922357 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482ff85-c759-4dcf-a0ec-d8880dd9ccda admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:27.922709 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482ff85-c759-4dcf-a0ec-d8880dd9ccda admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:27.923462 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5482ff85-c759-4dcf-a0ec-d8880dd9ccda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:27.925428 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5482ff85-c759-4dcf-a0ec-d8880dd9ccda admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:28.010815 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1294/2587] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:14:27 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:14:28.014586 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a2e06-5dd5-4719-bb4c-07902337ca3c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:28.024923 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a2e06-5dd5-4719-bb4c-07902337ca3c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:28.084990 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317881e-3e59-421e-9da8-5caa1e073485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:28.085900 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317881e-3e59-421e-9da8-5caa1e073485 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:28.086073 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317881e-3e59-421e-9da8-5caa1e073485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:28.086353 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317881e-3e59-421e-9da8-5caa1e073485 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:28.087054 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c317881e-3e59-421e-9da8-5caa1e073485 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:28.089053 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c317881e-3e59-421e-9da8-5caa1e073485 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:28.203898 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1294/2588] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:14:28 2022] POST /identity/v3/users => generated 286 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:14:28.208532 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482ff85-c759-4dcf-a0ec-d8880dd9ccda admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:28.219663 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482ff85-c759-4dcf-a0ec-d8880dd9ccda admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:28.225008 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4acd35-f323-4263-b8e3-40b662e84d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:28.225588 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4acd35-f323-4263-b8e3-40b662e84d02 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:28.225870 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4acd35-f323-4263-b8e3-40b662e84d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:28.226148 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4acd35-f323-4263-b8e3-40b662e84d02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:28.226888 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4acd35-f323-4263-b8e3-40b662e84d02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:28.228545 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4acd35-f323-4263-b8e3-40b662e84d02 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:28.239934 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1295/2589] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:14:28.244090 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317881e-3e59-421e-9da8-5caa1e073485 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:28.254688 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317881e-3e59-421e-9da8-5caa1e073485 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:28.259508 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dfa210-a110-49f8-a01d-22a5ccd003d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:28.260167 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfa210-a110-49f8-a01d-22a5ccd003d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:28.260551 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfa210-a110-49f8-a01d-22a5ccd003d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:28.260823 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfa210-a110-49f8-a01d-22a5ccd003d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:28.261544 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dfa210-a110-49f8-a01d-22a5ccd003d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:28.263146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dfa210-a110-49f8-a01d-22a5ccd003d2 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:28.273729 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1295/2590] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:14:28.277909 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4acd35-f323-4263-b8e3-40b662e84d02 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:28.288750 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4acd35-f323-4263-b8e3-40b662e84d02 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:28.294564 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc53713-df97-447a-ad77-28ee6d5d4e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:28.295655 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc53713-df97-447a-ad77-28ee6d5d4e69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:28.296045 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc53713-df97-447a-ad77-28ee6d5d4e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:28.296743 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc53713-df97-447a-ad77-28ee6d5d4e69 admin admin] PATH_INFO: `/v3/projects/46dd9b4a3b69447a86ddfed819723535/users/85f3f4f0347740288d2fa3ba6e154849/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:28.300644 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:28.313786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc53713-df97-447a-ad77-28ee6d5d4e69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46dd9b4a3b69447a86ddfed819723535, user_id=85f3f4f0347740288d2fa3ba6e154849, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:28.316558 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc53713-df97-447a-ad77-28ee6d5d4e69 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:28.381179 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1296/2591] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:14:28 2022] PUT /identity/v3/projects/46dd9b4a3b69447a86ddfed819723535/users/85f3f4f0347740288d2fa3ba6e154849/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:28.393797 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9a927-4711-4038-9b3f-aed58d83f61a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:28.394092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9a927-4711-4038-9b3f-aed58d83f61a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:28.394397 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9a927-4711-4038-9b3f-aed58d83f61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:28.422180 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-31c9a927-4711-4038-9b3f-aed58d83f61a None None] MFA Rules not processed for user `85f3f4f0347740288d2fa3ba6e154849`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:28.423414 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-31c9a927-4711-4038-9b3f-aed58d83f61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:28.469632 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:28.490349 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1296/2592] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:28.494774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc53713-df97-447a-ad77-28ee6d5d4e69 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:28.558160 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc53713-df97-447a-ad77-28ee6d5d4e69 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:28.563689 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b4f4e-59e4-48f4-a3b9-fd27b4f31eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:28.567313 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b4f4e-59e4-48f4-a3b9-fd27b4f31eec admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:28.567313 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b4f4e-59e4-48f4-a3b9-fd27b4f31eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:28.567313 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b4f4e-59e4-48f4-a3b9-fd27b4f31eec admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:28.567313 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b4f4e-59e4-48f4-a3b9-fd27b4f31eec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:28.567671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b4f4e-59e4-48f4-a3b9-fd27b4f31eec admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:28.643768 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1297/2593] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:14:28 2022] POST /identity/v3/projects => generated 371 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:14:28.647696 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9a927-4711-4038-9b3f-aed58d83f61a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:28.658453 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9a927-4711-4038-9b3f-aed58d83f61a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:28.715714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc005981-b9e7-4116-a0f1-ddb3ce1083dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:28.716412 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc005981-b9e7-4116-a0f1-ddb3ce1083dd admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:28.716701 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc005981-b9e7-4116-a0f1-ddb3ce1083dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:28.716974 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc005981-b9e7-4116-a0f1-ddb3ce1083dd admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:28.717654 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc005981-b9e7-4116-a0f1-ddb3ce1083dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:28.719588 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc005981-b9e7-4116-a0f1-ddb3ce1083dd admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:28.820638 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1297/2594] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:14:28 2022] POST /identity/v3/users => generated 292 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:14:28.824628 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b4f4e-59e4-48f4-a3b9-fd27b4f31eec admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:28.835767 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b4f4e-59e4-48f4-a3b9-fd27b4f31eec admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:28.840526 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f905f-a85d-43fd-bd18-e827b1df831e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:28.841174 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f905f-a85d-43fd-bd18-e827b1df831e admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:28.841453 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f905f-a85d-43fd-bd18-e827b1df831e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:28.841727 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f905f-a85d-43fd-bd18-e827b1df831e admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:28.842445 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f905f-a85d-43fd-bd18-e827b1df831e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:28.844141 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f905f-a85d-43fd-bd18-e827b1df831e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:28.861036 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1298/2595] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:14:28.865248 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc005981-b9e7-4116-a0f1-ddb3ce1083dd admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:28.876546 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc005981-b9e7-4116-a0f1-ddb3ce1083dd admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:28.881405 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e44396-99c3-4c5c-a2f9-2df829353799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:28.882294 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e44396-99c3-4c5c-a2f9-2df829353799 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:28.882578 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e44396-99c3-4c5c-a2f9-2df829353799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:28.882852 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e44396-99c3-4c5c-a2f9-2df829353799 admin admin] PATH_INFO: `/v3/projects/2c65ab6a62bd44f1bcc830489b26b887/users/1713f9b4cd864cb5bc8d5da63e6eff7c/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:28.898927 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e44396-99c3-4c5c-a2f9-2df829353799 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c65ab6a62bd44f1bcc830489b26b887, user_id=1713f9b4cd864cb5bc8d5da63e6eff7c, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:28.901653 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e44396-99c3-4c5c-a2f9-2df829353799 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:28.997731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1298/2596] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:14:28 2022] PUT /identity/v3/projects/2c65ab6a62bd44f1bcc830489b26b887/users/1713f9b4cd864cb5bc8d5da63e6eff7c/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:29.003799 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f905f-a85d-43fd-bd18-e827b1df831e admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:29.013168 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f905f-a85d-43fd-bd18-e827b1df831e admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:29.075235 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2321b-9056-4818-9a31-cdc21cc393f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:29.075911 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2321b-9056-4818-9a31-cdc21cc393f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:29.076219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2321b-9056-4818-9a31-cdc21cc393f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:29.076530 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2321b-9056-4818-9a31-cdc21cc393f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:29.077259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2321b-9056-4818-9a31-cdc21cc393f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:29.078863 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2321b-9056-4818-9a31-cdc21cc393f3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:29.089610 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1299/2597] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:14:29.093788 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e44396-99c3-4c5c-a2f9-2df829353799 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:29.104374 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e44396-99c3-4c5c-a2f9-2df829353799 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:29.109749 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8e008-9d04-42fd-9e3a-ad1fdf48dd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:29.110718 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8e008-9d04-42fd-9e3a-ad1fdf48dd93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:29.110997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8e008-9d04-42fd-9e3a-ad1fdf48dd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:29.111332 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8e008-9d04-42fd-9e3a-ad1fdf48dd93 admin admin] PATH_INFO: `/v3/projects/2c65ab6a62bd44f1bcc830489b26b887/users/1713f9b4cd864cb5bc8d5da63e6eff7c/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:29.113575 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8e008-9d04-42fd-9e3a-ad1fdf48dd93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c65ab6a62bd44f1bcc830489b26b887, user_id=1713f9b4cd864cb5bc8d5da63e6eff7c, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:29.116260 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8e008-9d04-42fd-9e3a-ad1fdf48dd93 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:29.127518 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1299/2598] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:14:29 2022] PUT /identity/v3/projects/2c65ab6a62bd44f1bcc830489b26b887/users/1713f9b4cd864cb5bc8d5da63e6eff7c/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:29.139058 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254c650-32c3-4a49-ac50-026047134fff None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:29.139351 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254c650-32c3-4a49-ac50-026047134fff None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:29.139711 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254c650-32c3-4a49-ac50-026047134fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:29.166525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6254c650-32c3-4a49-ac50-026047134fff None None] MFA Rules not processed for user `1713f9b4cd864cb5bc8d5da63e6eff7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:29.167768 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6254c650-32c3-4a49-ac50-026047134fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:29.213508 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:29.239869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1300/2599] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:29 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:29.251491 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8e008-9d04-42fd-9e3a-ad1fdf48dd93 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:29.253329 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3fa8e008-9d04-42fd-9e3a-ad1fdf48dd93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:29.317518 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8e008-9d04-42fd-9e3a-ad1fdf48dd93 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:29.321266 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448868b9-dea5-46af-b32a-80826d49841c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:29.321891 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448868b9-dea5-46af-b32a-80826d49841c service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:29.322176 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448868b9-dea5-46af-b32a-80826d49841c service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:29.322455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448868b9-dea5-46af-b32a-80826d49841c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:29.331603 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448868b9-dea5-46af-b32a-80826d49841c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:29.333353 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448868b9-dea5-46af-b32a-80826d49841c service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:29.343040 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1300/2600] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:14:29 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:14:30.231319 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254c650-32c3-4a49-ac50-026047134fff None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:30.293407 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254c650-32c3-4a49-ac50-026047134fff None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:30.298130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48690d1-c6fc-4160-84f9-a0c01b364362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:30.299235 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48690d1-c6fc-4160-84f9-a0c01b364362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:30.299521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48690d1-c6fc-4160-84f9-a0c01b364362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:30.299867 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48690d1-c6fc-4160-84f9-a0c01b364362 admin admin] PATH_INFO: `/v3/users/85f3f4f0347740288d2fa3ba6e154849` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:30.301180 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48690d1-c6fc-4160-84f9-a0c01b364362 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85f3f4f0347740288d2fa3ba6e154849)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:30.303521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48690d1-c6fc-4160-84f9-a0c01b364362 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:30.515921 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d48690d1-c6fc-4160-84f9-a0c01b364362 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85f3f4f0347740288d2fa3ba6e154849'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:30.516509 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d48690d1-c6fc-4160-84f9-a0c01b364362 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85f3f4f0347740288d2fa3ba6e154849'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:30.528486 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d48690d1-c6fc-4160-84f9-a0c01b364362 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85f3f4f0347740288d2fa3ba6e154849'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:30.535291 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d48690d1-c6fc-4160-84f9-a0c01b364362 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85f3f4f0347740288d2fa3ba6e154849'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:30.597268 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d48690d1-c6fc-4160-84f9-a0c01b364362 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85f3f4f0347740288d2fa3ba6e154849'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:30.603995 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1301/2601] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:14:30 2022] DELETE /identity/v3/users/85f3f4f0347740288d2fa3ba6e154849 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:30.608731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471de1e-bc42-425f-a5ff-4a9f45b03574 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:30.609745 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471de1e-bc42-425f-a5ff-4a9f45b03574 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:30.610018 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471de1e-bc42-425f-a5ff-4a9f45b03574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:30.627758 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-0471de1e-bc42-425f-a5ff-4a9f45b03574 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:30.628970 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0471de1e-bc42-425f-a5ff-4a9f45b03574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:30.663934 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:30.693777 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1301/2602] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:14:30.703208 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48690d1-c6fc-4160-84f9-a0c01b364362 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:30.704719 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d48690d1-c6fc-4160-84f9-a0c01b364362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:30.764954 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48690d1-c6fc-4160-84f9-a0c01b364362 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:30.768609 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b9be1-fcc0-465c-bc75-79c229c8cfcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:30.769191 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b9be1-fcc0-465c-bc75-79c229c8cfcd service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:30.769475 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b9be1-fcc0-465c-bc75-79c229c8cfcd service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:30.769751 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b9be1-fcc0-465c-bc75-79c229c8cfcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:30.779537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b9be1-fcc0-465c-bc75-79c229c8cfcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:30.781537 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b9be1-fcc0-465c-bc75-79c229c8cfcd service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:30.791424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1302/2603] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:14:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:14:31.179950 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471de1e-bc42-425f-a5ff-4a9f45b03574 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:31.181455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0471de1e-bc42-425f-a5ff-4a9f45b03574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:31.193853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0471de1e-bc42-425f-a5ff-4a9f45b03574 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:31.199242 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088549fa-e706-4f58-ba6f-f3053de5097f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:31.200213 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088549fa-e706-4f58-ba6f-f3053de5097f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:31.200529 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088549fa-e706-4f58-ba6f-f3053de5097f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:31.200811 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088549fa-e706-4f58-ba6f-f3053de5097f admin admin] PATH_INFO: `/v3/projects/46dd9b4a3b69447a86ddfed819723535` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:31.202214 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088549fa-e706-4f58-ba6f-f3053de5097f admin admin] RBAC: Authorizing `identity:delete_project(project_id=46dd9b4a3b69447a86ddfed819723535)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:31.204702 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088549fa-e706-4f58-ba6f-f3053de5097f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:31.322747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-088549fa-e706-4f58-ba6f-f3053de5097f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46dd9b4a3b69447a86ddfed819723535', 'request_id': 'req-088549fa-e706-4f58-ba6f-f3053de5097f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:31.385817 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-088549fa-e706-4f58-ba6f-f3053de5097f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46dd9b4a3b69447a86ddfed819723535', 'request_id': 'req-088549fa-e706-4f58-ba6f-f3053de5097f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:31.392646 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-088549fa-e706-4f58-ba6f-f3053de5097f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46dd9b4a3b69447a86ddfed819723535', 'request_id': 'req-088549fa-e706-4f58-ba6f-f3053de5097f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:31.403580 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-088549fa-e706-4f58-ba6f-f3053de5097f admin admin] The token cache is being invalidate because project 46dd9b4a3b69447a86ddfed819723535 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:14:31.404012 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-088549fa-e706-4f58-ba6f-f3053de5097f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:31.405739 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-088549fa-e706-4f58-ba6f-f3053de5097f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:31.411908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1302/2604] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:14:31 2022] DELETE /identity/v3/projects/46dd9b4a3b69447a86ddfed819723535 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:14:31.413675 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b9be1-fcc0-465c-bc75-79c229c8cfcd service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:31.418654 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:31.423898 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b9be1-fcc0-465c-bc75-79c229c8cfcd service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:31.479954 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36495e00-4bd3-409f-a9b1-e88a45ba48b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:31.480859 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36495e00-4bd3-409f-a9b1-e88a45ba48b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:31.481142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36495e00-4bd3-409f-a9b1-e88a45ba48b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:31.481426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36495e00-4bd3-409f-a9b1-e88a45ba48b6 admin admin] PATH_INFO: `/v3/users/1713f9b4cd864cb5bc8d5da63e6eff7c` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:31.482604 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36495e00-4bd3-409f-a9b1-e88a45ba48b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1713f9b4cd864cb5bc8d5da63e6eff7c)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:31.484582 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36495e00-4bd3-409f-a9b1-e88a45ba48b6 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:31.702163 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-36495e00-4bd3-409f-a9b1-e88a45ba48b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1713f9b4cd864cb5bc8d5da63e6eff7c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:31.702736 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-36495e00-4bd3-409f-a9b1-e88a45ba48b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1713f9b4cd864cb5bc8d5da63e6eff7c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:31.715337 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-36495e00-4bd3-409f-a9b1-e88a45ba48b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1713f9b4cd864cb5bc8d5da63e6eff7c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:31.722141 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-36495e00-4bd3-409f-a9b1-e88a45ba48b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1713f9b4cd864cb5bc8d5da63e6eff7c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:31.785835 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-36495e00-4bd3-409f-a9b1-e88a45ba48b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1713f9b4cd864cb5bc8d5da63e6eff7c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:31.793863 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1303/2605] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:14:31 2022] DELETE /identity/v3/users/1713f9b4cd864cb5bc8d5da63e6eff7c => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:32.217992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088549fa-e706-4f58-ba6f-f3053de5097f admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:32.219093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-088549fa-e706-4f58-ba6f-f3053de5097f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:32.232660 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088549fa-e706-4f58-ba6f-f3053de5097f admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:32.300223 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaafb73-92d1-4f52-89e5-4aa496dbed95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:32.301205 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaafb73-92d1-4f52-89e5-4aa496dbed95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:32.301480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaafb73-92d1-4f52-89e5-4aa496dbed95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:32.301836 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaafb73-92d1-4f52-89e5-4aa496dbed95 admin admin] PATH_INFO: `/v3/projects/2c65ab6a62bd44f1bcc830489b26b887` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:32.303105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaafb73-92d1-4f52-89e5-4aa496dbed95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c65ab6a62bd44f1bcc830489b26b887)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:32.305128 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaafb73-92d1-4f52-89e5-4aa496dbed95 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:32.386779 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9eaafb73-92d1-4f52-89e5-4aa496dbed95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c65ab6a62bd44f1bcc830489b26b887', 'request_id': 'req-9eaafb73-92d1-4f52-89e5-4aa496dbed95'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:32.425724 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9eaafb73-92d1-4f52-89e5-4aa496dbed95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c65ab6a62bd44f1bcc830489b26b887', 'request_id': 'req-9eaafb73-92d1-4f52-89e5-4aa496dbed95'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:32.431920 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9eaafb73-92d1-4f52-89e5-4aa496dbed95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c65ab6a62bd44f1bcc830489b26b887', 'request_id': 'req-9eaafb73-92d1-4f52-89e5-4aa496dbed95'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:32.433087 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:32.443656 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9eaafb73-92d1-4f52-89e5-4aa496dbed95 admin admin] The token cache is being invalidate because project 2c65ab6a62bd44f1bcc830489b26b887 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:14:32.443936 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9eaafb73-92d1-4f52-89e5-4aa496dbed95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:32.444586 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9eaafb73-92d1-4f52-89e5-4aa496dbed95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:32.446123 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1303/2606] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:14:32 2022] DELETE /identity/v3/projects/2c65ab6a62bd44f1bcc830489b26b887 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:32.452590 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70302bc-56db-4685-8936-b31323077b3e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:32.453072 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70302bc-56db-4685-8936-b31323077b3e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:32.453188 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70302bc-56db-4685-8936-b31323077b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:32.473107 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b70302bc-56db-4685-8936-b31323077b3e None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:32.474285 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b70302bc-56db-4685-8936-b31323077b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:32.536234 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1304/2607] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:32.544715 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5842a7-5e49-40b1-8138-6ca911d216df None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:32.544998 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5842a7-5e49-40b1-8138-6ca911d216df None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:32.545270 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5842a7-5e49-40b1-8138-6ca911d216df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:32.563936 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ee5842a7-5e49-40b1-8138-6ca911d216df None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:32.565163 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ee5842a7-5e49-40b1-8138-6ca911d216df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:32.573227 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1304/2608] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:32.578163 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70302bc-56db-4685-8936-b31323077b3e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:32.588463 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70302bc-56db-4685-8936-b31323077b3e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:32.593076 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252ae10-5038-4ae4-92e9-0980090799e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:32.594224 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252ae10-5038-4ae4-92e9-0980090799e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:32.594502 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252ae10-5038-4ae4-92e9-0980090799e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:32.594769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e252ae10-5038-4ae4-92e9-0980090799e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:32.595489 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e252ae10-5038-4ae4-92e9-0980090799e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:14:32.595789 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e252ae10-5038-4ae4-92e9-0980090799e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:32.597414 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e252ae10-5038-4ae4-92e9-0980090799e1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:32.610024 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1305/2609] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:14:32.615224 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5842a7-5e49-40b1-8138-6ca911d216df None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:32.625033 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5842a7-5e49-40b1-8138-6ca911d216df None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:32.629649 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f7be5-1cc1-4f42-a66b-40958730f961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:32.630304 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f7be5-1cc1-4f42-a66b-40958730f961 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:32.630587 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f7be5-1cc1-4f42-a66b-40958730f961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:32.630864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f7be5-1cc1-4f42-a66b-40958730f961 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:32.631566 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f7be5-1cc1-4f42-a66b-40958730f961 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:32.633518 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f7be5-1cc1-4f42-a66b-40958730f961 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:32.713657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1305/2610] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:14:32 2022] POST /identity/v3/projects => generated 363 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:14:32.717821 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252ae10-5038-4ae4-92e9-0980090799e1 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:32.729533 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e252ae10-5038-4ae4-92e9-0980090799e1 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:32.734539 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:32.788325 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65939624-7b01-4a36-b9ac-1db018fab57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:32.789014 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65939624-7b01-4a36-b9ac-1db018fab57a admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:32.789300 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65939624-7b01-4a36-b9ac-1db018fab57a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:32.789578 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65939624-7b01-4a36-b9ac-1db018fab57a admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:32.790280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65939624-7b01-4a36-b9ac-1db018fab57a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:32.792223 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65939624-7b01-4a36-b9ac-1db018fab57a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:32.889580 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1306/2611] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:14:32 2022] POST /identity/v3/users => generated 282 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:14:32.894370 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f7be5-1cc1-4f42-a66b-40958730f961 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:32.904771 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f7be5-1cc1-4f42-a66b-40958730f961 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:32.910413 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e2a7c-ab71-48e3-b124-9bea4fc99799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:32.911064 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e2a7c-ab71-48e3-b124-9bea4fc99799 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:32.911344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e2a7c-ab71-48e3-b124-9bea4fc99799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:32.911619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e2a7c-ab71-48e3-b124-9bea4fc99799 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:32.912366 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e2a7c-ab71-48e3-b124-9bea4fc99799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:32.914105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e2a7c-ab71-48e3-b124-9bea4fc99799 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:32.925386 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1306/2612] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:14:32.929204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65939624-7b01-4a36-b9ac-1db018fab57a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:32.939470 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65939624-7b01-4a36-b9ac-1db018fab57a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:32.943990 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204db0af-9904-4fbf-9b59-f8c927761245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:32.944665 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204db0af-9904-4fbf-9b59-f8c927761245 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:32.944946 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204db0af-9904-4fbf-9b59-f8c927761245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:32.945217 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204db0af-9904-4fbf-9b59-f8c927761245 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:32.945935 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204db0af-9904-4fbf-9b59-f8c927761245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:32.947694 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204db0af-9904-4fbf-9b59-f8c927761245 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:32.960095 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1307/2613] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:14:32.964160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e2a7c-ab71-48e3-b124-9bea4fc99799 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:32.974943 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e2a7c-ab71-48e3-b124-9bea4fc99799 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:32.979599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae224e-60b6-40a5-8076-69a47354a182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:32.980448 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae224e-60b6-40a5-8076-69a47354a182 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:32.980729 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae224e-60b6-40a5-8076-69a47354a182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:32.981002 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ae224e-60b6-40a5-8076-69a47354a182 admin admin] PATH_INFO: `/v3/projects/0b9844f2f8c94d9f8229ff7fcfb7da95/users/ab5cc0abe90b441295d604adf9164e76/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:32.995399 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae224e-60b6-40a5-8076-69a47354a182 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b9844f2f8c94d9f8229ff7fcfb7da95, user_id=ab5cc0abe90b441295d604adf9164e76, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:32.998113 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ae224e-60b6-40a5-8076-69a47354a182 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:33.105261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1307/2614] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:14:32 2022] PUT /identity/v3/projects/0b9844f2f8c94d9f8229ff7fcfb7da95/users/ab5cc0abe90b441295d604adf9164e76/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:33.115374 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1e9d8-6af8-4e42-b89e-109d2e5fb694 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:33.115752 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1e9d8-6af8-4e42-b89e-109d2e5fb694 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:33.116034 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b1e9d8-6af8-4e42-b89e-109d2e5fb694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:33.142681 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-51b1e9d8-6af8-4e42-b89e-109d2e5fb694 None None] MFA Rules not processed for user `ab5cc0abe90b441295d604adf9164e76`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:33.143887 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-51b1e9d8-6af8-4e42-b89e-109d2e5fb694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:33.206426 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1308/2615] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:33 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:33.211254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae224e-60b6-40a5-8076-69a47354a182 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:33.272171 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ae224e-60b6-40a5-8076-69a47354a182 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:33.277242 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23b358-43c5-4975-a93e-6e746c93b97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:33.277898 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23b358-43c5-4975-a93e-6e746c93b97e admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:33.278182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23b358-43c5-4975-a93e-6e746c93b97e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:33.278455 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23b358-43c5-4975-a93e-6e746c93b97e admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:33.279135 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23b358-43c5-4975-a93e-6e746c93b97e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:33.281403 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23b358-43c5-4975-a93e-6e746c93b97e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:33.370464 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1308/2616] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:14:33 2022] POST /identity/v3/projects => generated 365 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:14:33.374153 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1e9d8-6af8-4e42-b89e-109d2e5fb694 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:33.385681 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b1e9d8-6af8-4e42-b89e-109d2e5fb694 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:33.441448 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abced318-ee99-4599-88ff-358d399d0fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:33.442103 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abced318-ee99-4599-88ff-358d399d0fd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:33.442380 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abced318-ee99-4599-88ff-358d399d0fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:33.442649 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abced318-ee99-4599-88ff-358d399d0fd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:33.443332 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abced318-ee99-4599-88ff-358d399d0fd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:33.445287 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abced318-ee99-4599-88ff-358d399d0fd3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:33.528934 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1309/2617] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:14:33 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:14:33.532648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23b358-43c5-4975-a93e-6e746c93b97e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:33.542991 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23b358-43c5-4975-a93e-6e746c93b97e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:33.548104 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b287b2-91fa-4e49-97dc-ce6f26d63e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:33.548785 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b287b2-91fa-4e49-97dc-ce6f26d63e01 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:33.549066 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b287b2-91fa-4e49-97dc-ce6f26d63e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:33.549335 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b287b2-91fa-4e49-97dc-ce6f26d63e01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:33.550104 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b287b2-91fa-4e49-97dc-ce6f26d63e01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:33.551704 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b287b2-91fa-4e49-97dc-ce6f26d63e01 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:33.562053 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1309/2618] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:14:33.566692 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abced318-ee99-4599-88ff-358d399d0fd3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:33.577424 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abced318-ee99-4599-88ff-358d399d0fd3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:33.582883 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a9b78-f50b-4b00-b169-a8b5231c7163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:33.583900 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a9b78-f50b-4b00-b169-a8b5231c7163 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:33.584184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a9b78-f50b-4b00-b169-a8b5231c7163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:33.584495 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a9b78-f50b-4b00-b169-a8b5231c7163 admin admin] PATH_INFO: `/v3/projects/c7b178ea674246478478099e87efea4b/users/45374212f9904d6ca620fda976491a0b/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:33.586702 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:33.599859 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a9b78-f50b-4b00-b169-a8b5231c7163 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7b178ea674246478478099e87efea4b, user_id=45374212f9904d6ca620fda976491a0b, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:33.602668 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a9b78-f50b-4b00-b169-a8b5231c7163 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:33.688189 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1310/2619] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:14:33 2022] PUT /identity/v3/projects/c7b178ea674246478478099e87efea4b/users/45374212f9904d6ca620fda976491a0b/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:33.692245 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b287b2-91fa-4e49-97dc-ce6f26d63e01 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:33.702463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b287b2-91fa-4e49-97dc-ce6f26d63e01 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:33.757543 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5246b87-d6fb-4341-9016-b6e6e122cc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:33.758199 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5246b87-d6fb-4341-9016-b6e6e122cc29 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:33.758485 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5246b87-d6fb-4341-9016-b6e6e122cc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:33.758764 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5246b87-d6fb-4341-9016-b6e6e122cc29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:33.759485 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5246b87-d6fb-4341-9016-b6e6e122cc29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:33.761133 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5246b87-d6fb-4341-9016-b6e6e122cc29 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:33.766220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:33.771032 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1310/2620] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:14:33.775181 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a9b78-f50b-4b00-b169-a8b5231c7163 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:33.786300 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a9b78-f50b-4b00-b169-a8b5231c7163 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:33.790895 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d6784-270b-46e7-8880-b49c8dff28a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:33.791754 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d6784-270b-46e7-8880-b49c8dff28a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:33.792033 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d6784-270b-46e7-8880-b49c8dff28a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:33.792336 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d6784-270b-46e7-8880-b49c8dff28a9 admin admin] PATH_INFO: `/v3/projects/c7b178ea674246478478099e87efea4b/users/45374212f9904d6ca620fda976491a0b/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:33.794326 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d6784-270b-46e7-8880-b49c8dff28a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7b178ea674246478478099e87efea4b, user_id=45374212f9904d6ca620fda976491a0b, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:33.797923 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d6784-270b-46e7-8880-b49c8dff28a9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:33.808068 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1311/2621] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:14:33 2022] PUT /identity/v3/projects/c7b178ea674246478478099e87efea4b/users/45374212f9904d6ca620fda976491a0b/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:14:33.818966 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b024b71-547b-48ad-9ca8-c95e261d11ec None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:33.819256 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b024b71-547b-48ad-9ca8-c95e261d11ec None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:33.819534 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b024b71-547b-48ad-9ca8-c95e261d11ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:33.846523 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8b024b71-547b-48ad-9ca8-c95e261d11ec None None] MFA Rules not processed for user `45374212f9904d6ca620fda976491a0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:33.847721 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8b024b71-547b-48ad-9ca8-c95e261d11ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:33.913569 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1311/2622] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:33 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:33.925373 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d6784-270b-46e7-8880-b49c8dff28a9 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:33.927203 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3f1d6784-270b-46e7-8880-b49c8dff28a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:33.985062 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d6784-270b-46e7-8880-b49c8dff28a9 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:33.988861 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535caff7-1936-4707-a1c8-fadd82d8ddfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:33.989613 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535caff7-1936-4707-a1c8-fadd82d8ddfd service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:33.989897 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535caff7-1936-4707-a1c8-fadd82d8ddfd service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:33.990170 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535caff7-1936-4707-a1c8-fadd82d8ddfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:33.999832 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535caff7-1936-4707-a1c8-fadd82d8ddfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:34.001308 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535caff7-1936-4707-a1c8-fadd82d8ddfd service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:34.011221 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1312/2623] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:14:33 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:14:35.012293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b024b71-547b-48ad-9ca8-c95e261d11ec None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:35.078494 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b024b71-547b-48ad-9ca8-c95e261d11ec None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:35.083331 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe090e7-5c8a-4382-bb0d-f1e0f5e185c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:35.084196 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe090e7-5c8a-4382-bb0d-f1e0f5e185c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:35.084623 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe090e7-5c8a-4382-bb0d-f1e0f5e185c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:35.084897 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe090e7-5c8a-4382-bb0d-f1e0f5e185c4 admin admin] PATH_INFO: `/v3/users/ab5cc0abe90b441295d604adf9164e76` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:35.086076 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe090e7-5c8a-4382-bb0d-f1e0f5e185c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab5cc0abe90b441295d604adf9164e76)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:35.088004 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe090e7-5c8a-4382-bb0d-f1e0f5e185c4 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:35.256046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1fe090e7-5c8a-4382-bb0d-f1e0f5e185c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab5cc0abe90b441295d604adf9164e76'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:35.256786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1fe090e7-5c8a-4382-bb0d-f1e0f5e185c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab5cc0abe90b441295d604adf9164e76'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:35.322498 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1fe090e7-5c8a-4382-bb0d-f1e0f5e185c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab5cc0abe90b441295d604adf9164e76'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:35.323192 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1fe090e7-5c8a-4382-bb0d-f1e0f5e185c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab5cc0abe90b441295d604adf9164e76'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:35.336004 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1fe090e7-5c8a-4382-bb0d-f1e0f5e185c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab5cc0abe90b441295d604adf9164e76'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:35.348725 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1312/2624] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:14:35 2022] DELETE /identity/v3/users/ab5cc0abe90b441295d604adf9164e76 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:35.353361 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d566a-6e46-4f54-aeac-1b48fef6c4cf None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:35.353869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d566a-6e46-4f54-aeac-1b48fef6c4cf None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:35.353978 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d566a-6e46-4f54-aeac-1b48fef6c4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:35.371513 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8e9d566a-6e46-4f54-aeac-1b48fef6c4cf None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:35.372740 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8e9d566a-6e46-4f54-aeac-1b48fef6c4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:35.432553 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1313/2625] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:35.442337 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe090e7-5c8a-4382-bb0d-f1e0f5e185c4 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:35.443730 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1fe090e7-5c8a-4382-bb0d-f1e0f5e185c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:35.450548 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:35.500810 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe090e7-5c8a-4382-bb0d-f1e0f5e185c4 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:35.504603 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e6aa6-fdc2-40b9-906d-06af9fae3da4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:35.505238 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e6aa6-fdc2-40b9-906d-06af9fae3da4 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:35.505548 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e6aa6-fdc2-40b9-906d-06af9fae3da4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:35.505822 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e6aa6-fdc2-40b9-906d-06af9fae3da4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:35.515406 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e6aa6-fdc2-40b9-906d-06af9fae3da4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:35.516823 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e6aa6-fdc2-40b9-906d-06af9fae3da4 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:35.526170 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1313/2626] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:14:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:14:35.936450 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d566a-6e46-4f54-aeac-1b48fef6c4cf None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:35.937858 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8e9d566a-6e46-4f54-aeac-1b48fef6c4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:35.950269 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d566a-6e46-4f54-aeac-1b48fef6c4cf None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:35.956261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c2903-1899-4959-be7a-f4abc6d40d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:35.957362 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c2903-1899-4959-be7a-f4abc6d40d9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:35.957645 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c2903-1899-4959-be7a-f4abc6d40d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:35.957920 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c2903-1899-4959-be7a-f4abc6d40d9b admin admin] PATH_INFO: `/v3/projects/0b9844f2f8c94d9f8229ff7fcfb7da95` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:35.960859 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c2903-1899-4959-be7a-f4abc6d40d9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b9844f2f8c94d9f8229ff7fcfb7da95)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:35.962655 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c2903-1899-4959-be7a-f4abc6d40d9b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:36.071402 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9a0c2903-1899-4959-be7a-f4abc6d40d9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b9844f2f8c94d9f8229ff7fcfb7da95', 'request_id': 'req-9a0c2903-1899-4959-be7a-f4abc6d40d9b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:36.077718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9a0c2903-1899-4959-be7a-f4abc6d40d9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b9844f2f8c94d9f8229ff7fcfb7da95', 'request_id': 'req-9a0c2903-1899-4959-be7a-f4abc6d40d9b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:36.084351 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9a0c2903-1899-4959-be7a-f4abc6d40d9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b9844f2f8c94d9f8229ff7fcfb7da95', 'request_id': 'req-9a0c2903-1899-4959-be7a-f4abc6d40d9b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:36.088417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:36.163354 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9a0c2903-1899-4959-be7a-f4abc6d40d9b admin admin] The token cache is being invalidate because project 0b9844f2f8c94d9f8229ff7fcfb7da95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:14:36.163764 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9a0c2903-1899-4959-be7a-f4abc6d40d9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:36.165217 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9a0c2903-1899-4959-be7a-f4abc6d40d9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:36.166775 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1314/2627] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:14:35 2022] DELETE /identity/v3/projects/0b9844f2f8c94d9f8229ff7fcfb7da95 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:36.170668 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e6aa6-fdc2-40b9-906d-06af9fae3da4 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:36.180829 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e6aa6-fdc2-40b9-906d-06af9fae3da4 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:36.237224 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c57e51-e1af-4e1e-bb5a-ce303b2dcfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:36.238100 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c57e51-e1af-4e1e-bb5a-ce303b2dcfe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:36.238376 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c57e51-e1af-4e1e-bb5a-ce303b2dcfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:36.238648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c57e51-e1af-4e1e-bb5a-ce303b2dcfe1 admin admin] PATH_INFO: `/v3/users/45374212f9904d6ca620fda976491a0b` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:36.240004 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c57e51-e1af-4e1e-bb5a-ce303b2dcfe1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45374212f9904d6ca620fda976491a0b)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:36.242875 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c57e51-e1af-4e1e-bb5a-ce303b2dcfe1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:36.451616 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d1c57e51-e1af-4e1e-bb5a-ce303b2dcfe1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45374212f9904d6ca620fda976491a0b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:36.452258 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d1c57e51-e1af-4e1e-bb5a-ce303b2dcfe1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45374212f9904d6ca620fda976491a0b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:36.557385 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d1c57e51-e1af-4e1e-bb5a-ce303b2dcfe1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45374212f9904d6ca620fda976491a0b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:36.558060 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d1c57e51-e1af-4e1e-bb5a-ce303b2dcfe1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45374212f9904d6ca620fda976491a0b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:36.571467 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d1c57e51-e1af-4e1e-bb5a-ce303b2dcfe1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45374212f9904d6ca620fda976491a0b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:36.584053 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1314/2628] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:14:36 2022] DELETE /identity/v3/users/45374212f9904d6ca620fda976491a0b => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:37.312718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c2903-1899-4959-be7a-f4abc6d40d9b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:37.314709 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9a0c2903-1899-4959-be7a-f4abc6d40d9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:37.327652 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c2903-1899-4959-be7a-f4abc6d40d9b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:37.384496 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2d80b3-9500-4510-a213-5cb4117cfe83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:37.385358 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d80b3-9500-4510-a213-5cb4117cfe83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:37.385659 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d80b3-9500-4510-a213-5cb4117cfe83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:37.385915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d80b3-9500-4510-a213-5cb4117cfe83 admin admin] PATH_INFO: `/v3/projects/c7b178ea674246478478099e87efea4b` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:37.387095 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2d80b3-9500-4510-a213-5cb4117cfe83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7b178ea674246478478099e87efea4b)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:37.389052 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2d80b3-9500-4510-a213-5cb4117cfe83 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:37.494066 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0b2d80b3-9500-4510-a213-5cb4117cfe83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7b178ea674246478478099e87efea4b', 'request_id': 'req-0b2d80b3-9500-4510-a213-5cb4117cfe83'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:37.500383 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0b2d80b3-9500-4510-a213-5cb4117cfe83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7b178ea674246478478099e87efea4b', 'request_id': 'req-0b2d80b3-9500-4510-a213-5cb4117cfe83'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:37.505904 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0b2d80b3-9500-4510-a213-5cb4117cfe83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7b178ea674246478478099e87efea4b', 'request_id': 'req-0b2d80b3-9500-4510-a213-5cb4117cfe83'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:37.573196 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0b2d80b3-9500-4510-a213-5cb4117cfe83 admin admin] The token cache is being invalidate because project c7b178ea674246478478099e87efea4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:14:37.573487 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0b2d80b3-9500-4510-a213-5cb4117cfe83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:37.574108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0b2d80b3-9500-4510-a213-5cb4117cfe83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:37.575641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1315/2629] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:14:37 2022] DELETE /identity/v3/projects/c7b178ea674246478478099e87efea4b => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:37.580556 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7b6e8-83a9-4194-b08b-bdad92ae4051 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:37.583745 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7b6e8-83a9-4194-b08b-bdad92ae4051 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:37.583745 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7b6e8-83a9-4194-b08b-bdad92ae4051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:37.599429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-70b7b6e8-83a9-4194-b08b-bdad92ae4051 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:37.600653 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-70b7b6e8-83a9-4194-b08b-bdad92ae4051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:37.622298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:37.659656 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1315/2630] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:37.667464 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a54a7-eff3-48d2-bdf3-1f392aa607ae None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:37.667756 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a54a7-eff3-48d2-bdf3-1f392aa607ae None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:37.668050 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a54a7-eff3-48d2-bdf3-1f392aa607ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:37.687144 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e14a54a7-eff3-48d2-bdf3-1f392aa607ae None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:37.688472 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e14a54a7-eff3-48d2-bdf3-1f392aa607ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:37.696279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1316/2631] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:37.700550 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7b6e8-83a9-4194-b08b-bdad92ae4051 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:37.710422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7b6e8-83a9-4194-b08b-bdad92ae4051 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:37.715280 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb13d0-4569-459f-83b6-e43dd31fd459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:37.715916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb13d0-4569-459f-83b6-e43dd31fd459 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:37.716530 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb13d0-4569-459f-83b6-e43dd31fd459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:37.716530 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb13d0-4569-459f-83b6-e43dd31fd459 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:37.717201 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb13d0-4569-459f-83b6-e43dd31fd459 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:14:37.717503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb13d0-4569-459f-83b6-e43dd31fd459 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:37.719610 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb13d0-4569-459f-83b6-e43dd31fd459 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:37.731287 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1316/2632] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:14:37.736037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a54a7-eff3-48d2-bdf3-1f392aa607ae None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:37.745171 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14a54a7-eff3-48d2-bdf3-1f392aa607ae None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:37.749695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d18c45-3524-4f77-83e7-90d0808b009a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:37.750338 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d18c45-3524-4f77-83e7-90d0808b009a admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:37.750619 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d18c45-3524-4f77-83e7-90d0808b009a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:37.750897 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d18c45-3524-4f77-83e7-90d0808b009a admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:37.751593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d18c45-3524-4f77-83e7-90d0808b009a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:37.753531 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d18c45-3524-4f77-83e7-90d0808b009a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:37.826428 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1317/2633] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:14:37 2022] POST /identity/v3/projects => generated 377 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:14:37.831047 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb13d0-4569-459f-83b6-e43dd31fd459 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:37.841689 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb13d0-4569-459f-83b6-e43dd31fd459 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:37.897283 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ffb46-5251-4794-9b36-2ad5f3dd1075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:37.897882 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ffb46-5251-4794-9b36-2ad5f3dd1075 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:37.898303 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ffb46-5251-4794-9b36-2ad5f3dd1075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:37.898479 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ffb46-5251-4794-9b36-2ad5f3dd1075 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:37.899150 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ffb46-5251-4794-9b36-2ad5f3dd1075 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:37.901124 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ffb46-5251-4794-9b36-2ad5f3dd1075 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:37.980166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1317/2634] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:14:37 2022] POST /identity/v3/users => generated 289 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:14:37.984435 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d18c45-3524-4f77-83e7-90d0808b009a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:37.995137 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d18c45-3524-4f77-83e7-90d0808b009a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:38.002280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304a65c-009e-436d-8ab2-bf68600d4ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:38.002280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304a65c-009e-436d-8ab2-bf68600d4ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:38.002280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304a65c-009e-436d-8ab2-bf68600d4ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:38.002280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304a65c-009e-436d-8ab2-bf68600d4ee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:38.002679 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7304a65c-009e-436d-8ab2-bf68600d4ee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:38.004019 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7304a65c-009e-436d-8ab2-bf68600d4ee3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:38.014552 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1318/2635] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:14:38.018938 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ffb46-5251-4794-9b36-2ad5f3dd1075 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:38.028797 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ffb46-5251-4794-9b36-2ad5f3dd1075 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:38.034093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863b9a6-c153-410e-b861-6fcbe2853c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:38.034760 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863b9a6-c153-410e-b861-6fcbe2853c75 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:38.035040 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863b9a6-c153-410e-b861-6fcbe2853c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:38.035309 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863b9a6-c153-410e-b861-6fcbe2853c75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:38.036025 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2863b9a6-c153-410e-b861-6fcbe2853c75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:38.037723 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2863b9a6-c153-410e-b861-6fcbe2853c75 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:38.047557 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1318/2636] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:14:38.051660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304a65c-009e-436d-8ab2-bf68600d4ee3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:38.061655 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304a65c-009e-436d-8ab2-bf68600d4ee3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:38.067026 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbacab9-0f87-4d52-b1d7-12e75f8d78e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:38.067902 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbacab9-0f87-4d52-b1d7-12e75f8d78e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:38.068209 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbacab9-0f87-4d52-b1d7-12e75f8d78e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:38.068529 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbacab9-0f87-4d52-b1d7-12e75f8d78e7 admin admin] PATH_INFO: `/v3/projects/c80a565f133144289444653bfbbea7d9/users/d552dec700e543d6b79d989744475d97/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:38.084343 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbacab9-0f87-4d52-b1d7-12e75f8d78e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c80a565f133144289444653bfbbea7d9, user_id=d552dec700e543d6b79d989744475d97, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:38.087090 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbacab9-0f87-4d52-b1d7-12e75f8d78e7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:38.187422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1319/2637] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:14:38 2022] PUT /identity/v3/projects/c80a565f133144289444653bfbbea7d9/users/d552dec700e543d6b79d989744475d97/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:38.198277 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a8aab-c4fc-417d-be51-19e30a420bf3 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:38.198639 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a8aab-c4fc-417d-be51-19e30a420bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:38.198846 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a8aab-c4fc-417d-be51-19e30a420bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:38.227894 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ac4a8aab-c4fc-417d-be51-19e30a420bf3 None None] MFA Rules not processed for user `d552dec700e543d6b79d989744475d97`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:38.229158 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ac4a8aab-c4fc-417d-be51-19e30a420bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:38.305614 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1319/2638] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:38.309350 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbacab9-0f87-4d52-b1d7-12e75f8d78e7 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:38.337766 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:38.374546 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbacab9-0f87-4d52-b1d7-12e75f8d78e7 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:38.379795 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e6ca3-84df-4d2c-a1fd-6a7b4c7d935b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:38.380480 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e6ca3-84df-4d2c-a1fd-6a7b4c7d935b admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:38.380764 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e6ca3-84df-4d2c-a1fd-6a7b4c7d935b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:38.381035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e6ca3-84df-4d2c-a1fd-6a7b4c7d935b admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:38.381730 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e6ca3-84df-4d2c-a1fd-6a7b4c7d935b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:38.383618 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e6ca3-84df-4d2c-a1fd-6a7b4c7d935b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:38.493771 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1320/2639] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:14:38 2022] POST /identity/v3/projects => generated 377 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:14:38.497576 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a8aab-c4fc-417d-be51-19e30a420bf3 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:38.508993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a8aab-c4fc-417d-be51-19e30a420bf3 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:38.527550 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:38.565209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d175695-7aff-4d9a-b350-b509d341e6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:38.565911 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d175695-7aff-4d9a-b350-b509d341e6bf admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:38.566190 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d175695-7aff-4d9a-b350-b509d341e6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:38.566459 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d175695-7aff-4d9a-b350-b509d341e6bf admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:38.567134 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d175695-7aff-4d9a-b350-b509d341e6bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:38.569435 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d175695-7aff-4d9a-b350-b509d341e6bf admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:38.664276 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1320/2640] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:14:38 2022] POST /identity/v3/users => generated 295 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:14:38.668133 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e6ca3-84df-4d2c-a1fd-6a7b4c7d935b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:38.679172 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e6ca3-84df-4d2c-a1fd-6a7b4c7d935b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:38.683776 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9ddeb-ec44-4f97-8f4b-90f3c3a1abd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:38.684466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9ddeb-ec44-4f97-8f4b-90f3c3a1abd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:38.684822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9ddeb-ec44-4f97-8f4b-90f3c3a1abd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:38.685105 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9ddeb-ec44-4f97-8f4b-90f3c3a1abd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:38.685832 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9ddeb-ec44-4f97-8f4b-90f3c3a1abd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:38.687434 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf9ddeb-ec44-4f97-8f4b-90f3c3a1abd1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:38.698565 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1321/2641] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:14:38.702771 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d175695-7aff-4d9a-b350-b509d341e6bf admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:38.713457 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d175695-7aff-4d9a-b350-b509d341e6bf admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:38.717920 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70226fab-66e6-48af-9da3-e8ee42fb15ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:38.718776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70226fab-66e6-48af-9da3-e8ee42fb15ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:38.719051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70226fab-66e6-48af-9da3-e8ee42fb15ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:38.719317 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70226fab-66e6-48af-9da3-e8ee42fb15ea admin admin] PATH_INFO: `/v3/projects/8e367b73fd514479b58fec77d13c2766/users/ca44d02d4e0945aba537868796368365/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:38.733442 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70226fab-66e6-48af-9da3-e8ee42fb15ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e367b73fd514479b58fec77d13c2766, user_id=ca44d02d4e0945aba537868796368365, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:38.736663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70226fab-66e6-48af-9da3-e8ee42fb15ea admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:38.802478 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1321/2642] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:14:38 2022] PUT /identity/v3/projects/8e367b73fd514479b58fec77d13c2766/users/ca44d02d4e0945aba537868796368365/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:38.806271 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9ddeb-ec44-4f97-8f4b-90f3c3a1abd1 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:38.816709 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9ddeb-ec44-4f97-8f4b-90f3c3a1abd1 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:38.879219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a987614-2962-430b-9c5b-7572bc973611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:38.879894 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a987614-2962-430b-9c5b-7572bc973611 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:38.880206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a987614-2962-430b-9c5b-7572bc973611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:38.880492 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a987614-2962-430b-9c5b-7572bc973611 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:38.881227 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a987614-2962-430b-9c5b-7572bc973611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:38.882845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a987614-2962-430b-9c5b-7572bc973611 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:38.900397 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1322/2643] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:14:38.904538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70226fab-66e6-48af-9da3-e8ee42fb15ea admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:38.914765 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70226fab-66e6-48af-9da3-e8ee42fb15ea admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:38.919349 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911577ab-c522-4a1f-93c7-e5e10a27d852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:38.920210 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911577ab-c522-4a1f-93c7-e5e10a27d852 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:38.920524 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911577ab-c522-4a1f-93c7-e5e10a27d852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:38.920808 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911577ab-c522-4a1f-93c7-e5e10a27d852 admin admin] PATH_INFO: `/v3/projects/8e367b73fd514479b58fec77d13c2766/users/ca44d02d4e0945aba537868796368365/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:38.922796 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911577ab-c522-4a1f-93c7-e5e10a27d852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e367b73fd514479b58fec77d13c2766, user_id=ca44d02d4e0945aba537868796368365, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:38.926027 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911577ab-c522-4a1f-93c7-e5e10a27d852 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:38.936247 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1322/2644] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:14:38 2022] PUT /identity/v3/projects/8e367b73fd514479b58fec77d13c2766/users/ca44d02d4e0945aba537868796368365/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:38.947887 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8f671-bac8-468a-b619-90af7e347d20 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:38.948182 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8f671-bac8-468a-b619-90af7e347d20 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:38.948505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8f671-bac8-468a-b619-90af7e347d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:38.975129 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-bef8f671-bac8-468a-b619-90af7e347d20 None None] MFA Rules not processed for user `ca44d02d4e0945aba537868796368365`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:38.976395 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-bef8f671-bac8-468a-b619-90af7e347d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:39.045192 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1323/2645] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:38 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:39.056355 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911577ab-c522-4a1f-93c7-e5e10a27d852 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:39.058051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-911577ab-c522-4a1f-93c7-e5e10a27d852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:39.116482 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911577ab-c522-4a1f-93c7-e5e10a27d852 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:39.119928 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa4f52-b5b9-49f9-a3c8-1cbe53ddf68a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:39.120601 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa4f52-b5b9-49f9-a3c8-1cbe53ddf68a service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:39.120878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa4f52-b5b9-49f9-a3c8-1cbe53ddf68a service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:39.121145 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa4f52-b5b9-49f9-a3c8-1cbe53ddf68a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:39.174754 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa4f52-b5b9-49f9-a3c8-1cbe53ddf68a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:39.176184 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa4f52-b5b9-49f9-a3c8-1cbe53ddf68a service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:39.184790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1323/2646] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:14:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:14:45.487529 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8f671-bac8-468a-b619-90af7e347d20 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:45.501279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-bef8f671-bac8-468a-b619-90af7e347d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:45.501279 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8f671-bac8-468a-b619-90af7e347d20 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:45.504451 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc002be-3b15-40bc-b432-26a6cacf9d01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:45.505093 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc002be-3b15-40bc-b432-26a6cacf9d01 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:45.505381 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc002be-3b15-40bc-b432-26a6cacf9d01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:45.505655 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc002be-3b15-40bc-b432-26a6cacf9d01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:45.514698 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc002be-3b15-40bc-b432-26a6cacf9d01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:45.516132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc002be-3b15-40bc-b432-26a6cacf9d01 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:45.526486 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1324/2647] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:14:45 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:14:49.180871 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:55.521886 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:14:59.423373 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa4f52-b5b9-49f9-a3c8-1cbe53ddf68a service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:59.424864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-75aa4f52-b5b9-49f9-a3c8-1cbe53ddf68a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:59.493470 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa4f52-b5b9-49f9-a3c8-1cbe53ddf68a service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:59.493825 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d631184-9f88-47b8-a6d7-e0bd042a2e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:59.494516 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d631184-9f88-47b8-a6d7-e0bd042a2e41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:59.494795 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d631184-9f88-47b8-a6d7-e0bd042a2e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:59.495066 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d631184-9f88-47b8-a6d7-e0bd042a2e41 admin admin] PATH_INFO: `/v3/users/7b559ee39b7d4624b81791e9f66e9e72` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:59.496248 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d631184-9f88-47b8-a6d7-e0bd042a2e41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b559ee39b7d4624b81791e9f66e9e72)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:59.498192 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d631184-9f88-47b8-a6d7-e0bd042a2e41 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:59.683815 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9d631184-9f88-47b8-a6d7-e0bd042a2e41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b559ee39b7d4624b81791e9f66e9e72'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:59.684492 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9d631184-9f88-47b8-a6d7-e0bd042a2e41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b559ee39b7d4624b81791e9f66e9e72'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:59.768069 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9d631184-9f88-47b8-a6d7-e0bd042a2e41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b559ee39b7d4624b81791e9f66e9e72'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:59.768711 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9d631184-9f88-47b8-a6d7-e0bd042a2e41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b559ee39b7d4624b81791e9f66e9e72'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:59.781869 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9d631184-9f88-47b8-a6d7-e0bd042a2e41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b559ee39b7d4624b81791e9f66e9e72'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:14:59.795045 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1324/2648] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:14:59 2022] DELETE /identity/v3/users/7b559ee39b7d4624b81791e9f66e9e72 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:14:59.799680 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947902f-3b18-4d62-8b01-5a3f698b1a4a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:59.804551 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947902f-3b18-4d62-8b01-5a3f698b1a4a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:59.804551 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947902f-3b18-4d62-8b01-5a3f698b1a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:59.817341 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-2947902f-3b18-4d62-8b01-5a3f698b1a4a None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:14:59.818667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2947902f-3b18-4d62-8b01-5a3f698b1a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:59.881015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1325/2649] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:14:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:14:59.888068 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d631184-9f88-47b8-a6d7-e0bd042a2e41 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:14:59.889489 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9d631184-9f88-47b8-a6d7-e0bd042a2e41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:14:59.952762 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d631184-9f88-47b8-a6d7-e0bd042a2e41 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:14:59.957934 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adf3c8f-c8de-4db1-a01a-4ab4e710c987 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:14:59.958578 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf3c8f-c8de-4db1-a01a-4ab4e710c987 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:14:59.958855 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf3c8f-c8de-4db1-a01a-4ab4e710c987 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:14:59.959119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf3c8f-c8de-4db1-a01a-4ab4e710c987 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:14:59.968836 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adf3c8f-c8de-4db1-a01a-4ab4e710c987 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:14:59.972997 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adf3c8f-c8de-4db1-a01a-4ab4e710c987 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:14:59.980826 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1325/2650] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:14:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:15:00.404854 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947902f-3b18-4d62-8b01-5a3f698b1a4a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:00.406481 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2947902f-3b18-4d62-8b01-5a3f698b1a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:00.418464 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947902f-3b18-4d62-8b01-5a3f698b1a4a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:00.423707 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf173f8-95e6-48fe-ae68-2eef93b1f0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:00.424604 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf173f8-95e6-48fe-ae68-2eef93b1f0db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:00.424920 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf173f8-95e6-48fe-ae68-2eef93b1f0db admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:00.425193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf173f8-95e6-48fe-ae68-2eef93b1f0db admin admin] PATH_INFO: `/v3/projects/cf33365cc53d4914b29ad7448f8edd4a` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:00.427004 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf173f8-95e6-48fe-ae68-2eef93b1f0db admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf33365cc53d4914b29ad7448f8edd4a)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:00.429015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf173f8-95e6-48fe-ae68-2eef93b1f0db admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:00.539327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-dbf173f8-95e6-48fe-ae68-2eef93b1f0db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf33365cc53d4914b29ad7448f8edd4a', 'request_id': 'req-dbf173f8-95e6-48fe-ae68-2eef93b1f0db'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:00.546125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-dbf173f8-95e6-48fe-ae68-2eef93b1f0db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf33365cc53d4914b29ad7448f8edd4a', 'request_id': 'req-dbf173f8-95e6-48fe-ae68-2eef93b1f0db'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:00.551745 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-dbf173f8-95e6-48fe-ae68-2eef93b1f0db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf33365cc53d4914b29ad7448f8edd4a', 'request_id': 'req-dbf173f8-95e6-48fe-ae68-2eef93b1f0db'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:00.624730 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-dbf173f8-95e6-48fe-ae68-2eef93b1f0db admin admin] The token cache is being invalidate because project cf33365cc53d4914b29ad7448f8edd4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:15:00.625021 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-dbf173f8-95e6-48fe-ae68-2eef93b1f0db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:00.625643 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-dbf173f8-95e6-48fe-ae68-2eef93b1f0db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:00.627223 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1326/2651] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:15:00 2022] DELETE /identity/v3/projects/cf33365cc53d4914b29ad7448f8edd4a => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:15:00.632655 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1df548-3866-4ce0-9749-84a90b7d63a4 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:00.633124 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1df548-3866-4ce0-9749-84a90b7d63a4 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:00.633231 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1df548-3866-4ce0-9749-84a90b7d63a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:00.650883 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8e1df548-3866-4ce0-9749-84a90b7d63a4 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:15:00.652567 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8e1df548-3866-4ce0-9749-84a90b7d63a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:00.669346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:15:00.712722 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1326/2652] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:15:00.720962 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d769e-d145-4b57-9c7e-9b31bafdc3c2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:00.721251 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d769e-d145-4b57-9c7e-9b31bafdc3c2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:00.721533 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d769e-d145-4b57-9c7e-9b31bafdc3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:00.742266 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-517d769e-d145-4b57-9c7e-9b31bafdc3c2 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:15:00.743475 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-517d769e-d145-4b57-9c7e-9b31bafdc3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:00.751591 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1327/2653] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:15:00.756420 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1df548-3866-4ce0-9749-84a90b7d63a4 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:00.766285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1df548-3866-4ce0-9749-84a90b7d63a4 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:00.771518 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924be9d5-5866-4a90-9ade-9317e26b982b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:00.772167 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924be9d5-5866-4a90-9ade-9317e26b982b admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:00.772511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924be9d5-5866-4a90-9ade-9317e26b982b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:00.772786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924be9d5-5866-4a90-9ade-9317e26b982b admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:00.773536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924be9d5-5866-4a90-9ade-9317e26b982b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:15:00.773839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924be9d5-5866-4a90-9ade-9317e26b982b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:00.775504 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924be9d5-5866-4a90-9ade-9317e26b982b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:00.786833 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1327/2654] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:15:00.790923 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d769e-d145-4b57-9c7e-9b31bafdc3c2 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:00.800041 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517d769e-d145-4b57-9c7e-9b31bafdc3c2 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:00.804855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7467c91-057c-4afe-a58c-0301ff63d52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:00.805498 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7467c91-057c-4afe-a58c-0301ff63d52d admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:00.805784 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7467c91-057c-4afe-a58c-0301ff63d52d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:00.806061 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7467c91-057c-4afe-a58c-0301ff63d52d admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:00.806781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7467c91-057c-4afe-a58c-0301ff63d52d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:00.808723 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7467c91-057c-4afe-a58c-0301ff63d52d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:00.910310 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1328/2655] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:15:00 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:15:00.914384 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924be9d5-5866-4a90-9ade-9317e26b982b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:00.924524 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924be9d5-5866-4a90-9ade-9317e26b982b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:00.989723 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dad34c-36f4-455d-8c1f-968747f1321e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:00.990397 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dad34c-36f4-455d-8c1f-968747f1321e admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:00.990685 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dad34c-36f4-455d-8c1f-968747f1321e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:00.990960 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dad34c-36f4-455d-8c1f-968747f1321e admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:00.991656 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dad34c-36f4-455d-8c1f-968747f1321e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:00.993712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dad34c-36f4-455d-8c1f-968747f1321e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:01.065870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1328/2656] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:15:00 2022] POST /identity/v3/users => generated 293 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:15:01.070807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7467c91-057c-4afe-a58c-0301ff63d52d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:01.081283 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7467c91-057c-4afe-a58c-0301ff63d52d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:01.086149 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbef5f3-35cd-4fb4-9076-45b0ea4162d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:01.086818 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbef5f3-35cd-4fb4-9076-45b0ea4162d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:01.087138 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbef5f3-35cd-4fb4-9076-45b0ea4162d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:01.087421 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbef5f3-35cd-4fb4-9076-45b0ea4162d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:01.088151 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbef5f3-35cd-4fb4-9076-45b0ea4162d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:01.089795 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbef5f3-35cd-4fb4-9076-45b0ea4162d6 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:01.100151 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1329/2657] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:15:01.103988 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dad34c-36f4-455d-8c1f-968747f1321e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:01.114623 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dad34c-36f4-455d-8c1f-968747f1321e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:01.119298 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67722251-c2c0-4b0c-b6ca-671972b9522b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:01.119971 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67722251-c2c0-4b0c-b6ca-671972b9522b admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:01.120251 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67722251-c2c0-4b0c-b6ca-671972b9522b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:01.120560 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67722251-c2c0-4b0c-b6ca-671972b9522b admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:01.121277 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67722251-c2c0-4b0c-b6ca-671972b9522b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:01.122875 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67722251-c2c0-4b0c-b6ca-671972b9522b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:01.133154 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1329/2658] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:15:01.137122 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbef5f3-35cd-4fb4-9076-45b0ea4162d6 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:01.146858 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbef5f3-35cd-4fb4-9076-45b0ea4162d6 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:01.151471 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5b98e-c6a0-4389-b650-9da085709702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:01.152365 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5b98e-c6a0-4389-b650-9da085709702 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:01.152653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5b98e-c6a0-4389-b650-9da085709702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:01.152931 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5b98e-c6a0-4389-b650-9da085709702 admin admin] PATH_INFO: `/v3/projects/a2028a4056254026b31f412942d1f576/users/a9fe5380b3974516ac4e5b2fd9284348/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:01.167171 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a5b98e-c6a0-4389-b650-9da085709702 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2028a4056254026b31f412942d1f576, user_id=a9fe5380b3974516ac4e5b2fd9284348, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:01.169793 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a5b98e-c6a0-4389-b650-9da085709702 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:01.242481 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1330/2659] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:15:01 2022] PUT /identity/v3/projects/a2028a4056254026b31f412942d1f576/users/a9fe5380b3974516ac4e5b2fd9284348/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:15:01.247432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59009ff1-1b2f-45dc-a489-b1e00e944dea None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:01.247726 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59009ff1-1b2f-45dc-a489-b1e00e944dea None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:01.248003 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59009ff1-1b2f-45dc-a489-b1e00e944dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:01.265975 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-59009ff1-1b2f-45dc-a489-b1e00e944dea None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:15:01.267177 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-59009ff1-1b2f-45dc-a489-b1e00e944dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:01.326165 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1330/2660] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:15:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:15:01.336040 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5b98e-c6a0-4389-b650-9da085709702 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:01.337750 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-93a5b98e-c6a0-4389-b650-9da085709702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:01.359142 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:15:01.394815 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5b98e-c6a0-4389-b650-9da085709702 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:01.398564 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458dae1f-1134-4478-8d24-da80d8b22460 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:01.399203 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458dae1f-1134-4478-8d24-da80d8b22460 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:01.399487 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458dae1f-1134-4478-8d24-da80d8b22460 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:01.399763 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458dae1f-1134-4478-8d24-da80d8b22460 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:01.408659 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458dae1f-1134-4478-8d24-da80d8b22460 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:01.410624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458dae1f-1134-4478-8d24-da80d8b22460 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:01.420379 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1331/2661] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:15:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:15:06.678267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e944b9a-7538-4720-9530-8e4d6d88c98b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:06.678991 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e944b9a-7538-4720-9530-8e4d6d88c98b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:06.678991 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e944b9a-7538-4720-9530-8e4d6d88c98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:06.712473 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-5e944b9a-7538-4720-9530-8e4d6d88c98b None None] MFA Rules not processed for user `a9fe5380b3974516ac4e5b2fd9284348`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:15:06.717303 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5e944b9a-7538-4720-9530-8e4d6d88c98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:06.736971 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:15:06.787497 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1331/2662] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:15:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:15:06.792094 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458dae1f-1134-4478-8d24-da80d8b22460 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:06.803818 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458dae1f-1134-4478-8d24-da80d8b22460 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:06.809966 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfce08-1e5d-4794-a04b-76c6b3472128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:06.810776 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfce08-1e5d-4794-a04b-76c6b3472128 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:06.811063 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfce08-1e5d-4794-a04b-76c6b3472128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:06.811339 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfce08-1e5d-4794-a04b-76c6b3472128 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:06.812102 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddfce08-1e5d-4794-a04b-76c6b3472128 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:06.814065 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddfce08-1e5d-4794-a04b-76c6b3472128 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:06.965851 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1332/2663] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:15:06 2022] POST /identity/v3/projects => generated 383 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:15:06.973427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e944b9a-7538-4720-9530-8e4d6d88c98b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:06.984448 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e944b9a-7538-4720-9530-8e4d6d88c98b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:07.056917 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59698f-3494-476a-8524-9ff6c2bf9691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:07.057771 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59698f-3494-476a-8524-9ff6c2bf9691 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:07.058285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59698f-3494-476a-8524-9ff6c2bf9691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:07.058638 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59698f-3494-476a-8524-9ff6c2bf9691 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:07.060023 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f59698f-3494-476a-8524-9ff6c2bf9691 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:07.062599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f59698f-3494-476a-8524-9ff6c2bf9691 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:07.146482 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1332/2664] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:15:06 2022] POST /identity/v3/users => generated 298 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:15:07.150359 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfce08-1e5d-4794-a04b-76c6b3472128 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:07.161615 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfce08-1e5d-4794-a04b-76c6b3472128 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:07.166412 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779e72b-e5d1-45ff-8ecd-c63c1c15868f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:07.167132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779e72b-e5d1-45ff-8ecd-c63c1c15868f admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:07.167433 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779e72b-e5d1-45ff-8ecd-c63c1c15868f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:07.167624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779e72b-e5d1-45ff-8ecd-c63c1c15868f admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:07.168413 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4779e72b-e5d1-45ff-8ecd-c63c1c15868f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:07.170156 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4779e72b-e5d1-45ff-8ecd-c63c1c15868f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:07.184047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1333/2665] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:15:07.188056 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59698f-3494-476a-8524-9ff6c2bf9691 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:07.207068 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59698f-3494-476a-8524-9ff6c2bf9691 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:07.207068 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db776f0-b1c5-498f-8b12-35e8387286af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:07.207068 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db776f0-b1c5-498f-8b12-35e8387286af admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:07.207068 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db776f0-b1c5-498f-8b12-35e8387286af admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:07.207843 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db776f0-b1c5-498f-8b12-35e8387286af admin admin] PATH_INFO: `/v3/projects/29ed429db99d48b285a2e295ec0073bd/users/c24fddf0b70441c1947610ce9578c98c/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:07.229351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db776f0-b1c5-498f-8b12-35e8387286af admin admin] RBAC: Authorizing `identity:create_grant(project_id=29ed429db99d48b285a2e295ec0073bd, user_id=c24fddf0b70441c1947610ce9578c98c, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:07.232938 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db776f0-b1c5-498f-8b12-35e8387286af admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:07.361548 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779e72b-e5d1-45ff-8ecd-c63c1c15868f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:07.361995 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1333/2666] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:15:07 2022] PUT /identity/v3/projects/29ed429db99d48b285a2e295ec0073bd/users/c24fddf0b70441c1947610ce9578c98c/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:15:07.376189 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779e72b-e5d1-45ff-8ecd-c63c1c15868f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:07.455740 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65d53e0-f2af-4277-b04d-8447363aea97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:07.456511 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d53e0-f2af-4277-b04d-8447363aea97 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:07.456804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d53e0-f2af-4277-b04d-8447363aea97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:07.457192 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65d53e0-f2af-4277-b04d-8447363aea97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:07.458184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65d53e0-f2af-4277-b04d-8447363aea97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:07.459694 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65d53e0-f2af-4277-b04d-8447363aea97 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:07.470914 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1334/2667] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:15:07.475465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db776f0-b1c5-498f-8b12-35e8387286af admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:07.486952 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db776f0-b1c5-498f-8b12-35e8387286af admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:07.492933 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994be29-8af3-4091-a2fa-7cc29d74fac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:07.493880 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994be29-8af3-4091-a2fa-7cc29d74fac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:07.494157 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994be29-8af3-4091-a2fa-7cc29d74fac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:07.494428 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994be29-8af3-4091-a2fa-7cc29d74fac3 admin admin] PATH_INFO: `/v3/projects/29ed429db99d48b285a2e295ec0073bd/users/c24fddf0b70441c1947610ce9578c98c/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:07.496564 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d994be29-8af3-4091-a2fa-7cc29d74fac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29ed429db99d48b285a2e295ec0073bd, user_id=c24fddf0b70441c1947610ce9578c98c, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:07.499432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d994be29-8af3-4091-a2fa-7cc29d74fac3 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:07.512484 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1334/2668] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:15:07 2022] PUT /identity/v3/projects/29ed429db99d48b285a2e295ec0073bd/users/c24fddf0b70441c1947610ce9578c98c/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:15:13.430853 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585358a-d10e-4287-8dd4-95392193dcf7 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:13.431777 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585358a-d10e-4287-8dd4-95392193dcf7 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:13.432059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585358a-d10e-4287-8dd4-95392193dcf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:13.461710 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c585358a-d10e-4287-8dd4-95392193dcf7 None None] MFA Rules not processed for user `c24fddf0b70441c1947610ce9578c98c`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:15:13.463039 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c585358a-d10e-4287-8dd4-95392193dcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:13.536696 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1335/2669] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:15:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:15:13.548624 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994be29-8af3-4091-a2fa-7cc29d74fac3 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:13.550473 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d994be29-8af3-4091-a2fa-7cc29d74fac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:13.621203 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994be29-8af3-4091-a2fa-7cc29d74fac3 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:13.625756 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c555ebef-ee17-4de4-bf6f-69918dbcecc9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd72f952df34ad8853fe84673a41f45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:13.625951 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555ebef-ee17-4de4-bf6f-69918dbcecc9 service glance] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:13.626229 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555ebef-ee17-4de4-bf6f-69918dbcecc9 service glance] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:13.626580 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555ebef-ee17-4de4-bf6f-69918dbcecc9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:13.655981 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:15:13.690161 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c555ebef-ee17-4de4-bf6f-69918dbcecc9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:13.692566 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c555ebef-ee17-4de4-bf6f-69918dbcecc9 service glance] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:13.703253 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1335/2670] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:15:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:15:13.968201 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22da23a-f04d-4d66-868d-47e2f0b29dfc None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:13.968525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22da23a-f04d-4d66-868d-47e2f0b29dfc None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:13.968801 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22da23a-f04d-4d66-868d-47e2f0b29dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:13.986277 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e22da23a-f04d-4d66-868d-47e2f0b29dfc None None] MFA Rules not processed for user `1d37f369749143ce813c693adc5f2ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:15:13.987467 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e22da23a-f04d-4d66-868d-47e2f0b29dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:13.993528 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:15:14.048772 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1336/2671] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jun 23 20:15:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:15:14.213053 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c555ebef-ee17-4de4-bf6f-69918dbcecc9 service glance] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:14.216491 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c555ebef-ee17-4de4-bf6f-69918dbcecc9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:14.312586 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c555ebef-ee17-4de4-bf6f-69918dbcecc9 service glance] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:14.317613 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2755a-2e5c-4bdc-acc4-bbd0ef8558d9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96db03079816461bb322633f64b9fb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:14.318262 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2755a-2e5c-4bdc-acc4-bbd0ef8558d9 service swift] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:14.318545 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2755a-2e5c-4bdc-acc4-bbd0ef8558d9 service swift] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:14.318813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2755a-2e5c-4bdc-acc4-bbd0ef8558d9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:14.327118 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2755a-2e5c-4bdc-acc4-bbd0ef8558d9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:14.328771 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2755a-2e5c-4bdc-acc4-bbd0ef8558d9 service swift] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:14.337397 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1336/2672] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jun 23 20:15:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 20:15:15.123257 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22da23a-f04d-4d66-868d-47e2f0b29dfc None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:15.127446 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e22da23a-f04d-4d66-868d-47e2f0b29dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:15.206207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22da23a-f04d-4d66-868d-47e2f0b29dfc None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:15.211805 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb1c3d-7b20-4356-af19-f52389269ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:15.212823 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb1c3d-7b20-4356-af19-f52389269ad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:15.217073 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb1c3d-7b20-4356-af19-f52389269ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:15.217073 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb1c3d-7b20-4356-af19-f52389269ad2 admin admin] PATH_INFO: `/v3/users/d552dec700e543d6b79d989744475d97` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:15.217073 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb1c3d-7b20-4356-af19-f52389269ad2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d552dec700e543d6b79d989744475d97)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:15.217073 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb1c3d-7b20-4356-af19-f52389269ad2 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:15.463711 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0dfb1c3d-7b20-4356-af19-f52389269ad2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd552dec700e543d6b79d989744475d97'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:15.464406 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0dfb1c3d-7b20-4356-af19-f52389269ad2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd552dec700e543d6b79d989744475d97'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:15.492708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0dfb1c3d-7b20-4356-af19-f52389269ad2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd552dec700e543d6b79d989744475d97'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:15.506753 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0dfb1c3d-7b20-4356-af19-f52389269ad2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd552dec700e543d6b79d989744475d97'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:15.585479 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0dfb1c3d-7b20-4356-af19-f52389269ad2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd552dec700e543d6b79d989744475d97'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:15.596218 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1337/2673] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:15:15 2022] DELETE /identity/v3/users/d552dec700e543d6b79d989744475d97 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:15:15.597831 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17232908-d875-44c5-a37b-e7e448592469 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:15.598306 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17232908-d875-44c5-a37b-e7e448592469 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:15.598388 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17232908-d875-44c5-a37b-e7e448592469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:15.636524 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-17232908-d875-44c5-a37b-e7e448592469 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:15:15.637787 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-17232908-d875-44c5-a37b-e7e448592469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:15.820427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1337/2674] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:15:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:15:15.829253 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb1c3d-7b20-4356-af19-f52389269ad2 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:15.832509 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0dfb1c3d-7b20-4356-af19-f52389269ad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:15.930584 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:15:15.977324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb1c3d-7b20-4356-af19-f52389269ad2 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:15.981290 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ed579-11fa-4e9d-9f64-df4dcd127f7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:15.982006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ed579-11fa-4e9d-9f64-df4dcd127f7d service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:15.982336 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ed579-11fa-4e9d-9f64-df4dcd127f7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:15.982613 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ed579-11fa-4e9d-9f64-df4dcd127f7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:16.001289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ed579-11fa-4e9d-9f64-df4dcd127f7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:16.003926 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ed579-11fa-4e9d-9f64-df4dcd127f7d service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:16.042324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1338/2675] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:15:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:15:16.629877 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17232908-d875-44c5-a37b-e7e448592469 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:16.631372 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-17232908-d875-44c5-a37b-e7e448592469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:16.650941 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17232908-d875-44c5-a37b-e7e448592469 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:16.661491 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e59c0c-8c14-4e22-a634-52373ba52ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:16.662288 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e59c0c-8c14-4e22-a634-52373ba52ec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:16.662603 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e59c0c-8c14-4e22-a634-52373ba52ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:16.662878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e59c0c-8c14-4e22-a634-52373ba52ec5 admin admin] PATH_INFO: `/v3/projects/c80a565f133144289444653bfbbea7d9` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:16.671646 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e59c0c-8c14-4e22-a634-52373ba52ec5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c80a565f133144289444653bfbbea7d9)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:16.673624 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e59c0c-8c14-4e22-a634-52373ba52ec5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:16.809454 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a2e59c0c-8c14-4e22-a634-52373ba52ec5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c80a565f133144289444653bfbbea7d9', 'request_id': 'req-a2e59c0c-8c14-4e22-a634-52373ba52ec5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:16.811302 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:15:16.911589 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a2e59c0c-8c14-4e22-a634-52373ba52ec5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c80a565f133144289444653bfbbea7d9', 'request_id': 'req-a2e59c0c-8c14-4e22-a634-52373ba52ec5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:16.923034 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a2e59c0c-8c14-4e22-a634-52373ba52ec5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c80a565f133144289444653bfbbea7d9', 'request_id': 'req-a2e59c0c-8c14-4e22-a634-52373ba52ec5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:16.939774 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a2e59c0c-8c14-4e22-a634-52373ba52ec5 admin admin] The token cache is being invalidate because project c80a565f133144289444653bfbbea7d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:15:16.940099 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a2e59c0c-8c14-4e22-a634-52373ba52ec5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:16.940759 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a2e59c0c-8c14-4e22-a634-52373ba52ec5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:16.945763 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1338/2676] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:15:16 2022] DELETE /identity/v3/projects/c80a565f133144289444653bfbbea7d9 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:15:16.946633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ed579-11fa-4e9d-9f64-df4dcd127f7d service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:16.960665 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ed579-11fa-4e9d-9f64-df4dcd127f7d service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:17.051625 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e427393e-e9fd-4b14-a198-fe717d0844df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:17.052627 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427393e-e9fd-4b14-a198-fe717d0844df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:17.052912 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427393e-e9fd-4b14-a198-fe717d0844df admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:17.053186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427393e-e9fd-4b14-a198-fe717d0844df admin admin] PATH_INFO: `/v3/users/ca44d02d4e0945aba537868796368365` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:17.056580 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e427393e-e9fd-4b14-a198-fe717d0844df admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca44d02d4e0945aba537868796368365)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:17.058638 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e427393e-e9fd-4b14-a198-fe717d0844df admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:17.270839 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e427393e-e9fd-4b14-a198-fe717d0844df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca44d02d4e0945aba537868796368365'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:17.271571 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e427393e-e9fd-4b14-a198-fe717d0844df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca44d02d4e0945aba537868796368365'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:17.287701 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e427393e-e9fd-4b14-a198-fe717d0844df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca44d02d4e0945aba537868796368365'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:17.297118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e427393e-e9fd-4b14-a198-fe717d0844df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca44d02d4e0945aba537868796368365'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:17.373856 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e427393e-e9fd-4b14-a198-fe717d0844df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca44d02d4e0945aba537868796368365'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:17.382601 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1339/2677] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:15:16 2022] DELETE /identity/v3/users/ca44d02d4e0945aba537868796368365 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:15:17.975813 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e59c0c-8c14-4e22-a634-52373ba52ec5 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:17.977387 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a2e59c0c-8c14-4e22-a634-52373ba52ec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:17.992149 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e59c0c-8c14-4e22-a634-52373ba52ec5 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:18.053278 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4815c7-7761-400e-969f-32f9050af69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:18.054162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4815c7-7761-400e-969f-32f9050af69a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:18.054433 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4815c7-7761-400e-969f-32f9050af69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:18.054706 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4815c7-7761-400e-969f-32f9050af69a admin admin] PATH_INFO: `/v3/projects/8e367b73fd514479b58fec77d13c2766` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:18.056070 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4815c7-7761-400e-969f-32f9050af69a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e367b73fd514479b58fec77d13c2766)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:18.057974 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4815c7-7761-400e-969f-32f9050af69a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:18.194902 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9f4815c7-7761-400e-969f-32f9050af69a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e367b73fd514479b58fec77d13c2766', 'request_id': 'req-9f4815c7-7761-400e-969f-32f9050af69a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:18.271098 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9f4815c7-7761-400e-969f-32f9050af69a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e367b73fd514479b58fec77d13c2766', 'request_id': 'req-9f4815c7-7761-400e-969f-32f9050af69a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:18.281924 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9f4815c7-7761-400e-969f-32f9050af69a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e367b73fd514479b58fec77d13c2766', 'request_id': 'req-9f4815c7-7761-400e-969f-32f9050af69a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:18.298944 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9f4815c7-7761-400e-969f-32f9050af69a admin admin] The token cache is being invalidate because project 8e367b73fd514479b58fec77d13c2766 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:15:18.299154 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9f4815c7-7761-400e-969f-32f9050af69a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:18.299938 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9f4815c7-7761-400e-969f-32f9050af69a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:18.302160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1339/2678] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:15:17 2022] DELETE /identity/v3/projects/8e367b73fd514479b58fec77d13c2766 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:15:18.306754 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b926462-74c6-47ac-8015-626065c8db55 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:18.307210 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b926462-74c6-47ac-8015-626065c8db55 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:18.307315 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b926462-74c6-47ac-8015-626065c8db55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:18.328515 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9b926462-74c6-47ac-8015-626065c8db55 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:15:18.329737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9b926462-74c6-47ac-8015-626065c8db55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:18.393658 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:15:18.423708 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1340/2679] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:15:18.425156 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396002d-3f50-4570-a688-82ff21b2c641 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:18.425449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396002d-3f50-4570-a688-82ff21b2c641 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:18.425729 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7396002d-3f50-4570-a688-82ff21b2c641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:18.453398 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-7396002d-3f50-4570-a688-82ff21b2c641 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:15:18.454680 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7396002d-3f50-4570-a688-82ff21b2c641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:18.478815 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b926462-74c6-47ac-8015-626065c8db55 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:18.485697 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1340/2680] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:15:18.489965 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b926462-74c6-47ac-8015-626065c8db55 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:18.495379 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44b58b-2129-4464-b14f-dd9dcdae860a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:18.496145 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44b58b-2129-4464-b14f-dd9dcdae860a admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:18.496590 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44b58b-2129-4464-b14f-dd9dcdae860a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:18.496927 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44b58b-2129-4464-b14f-dd9dcdae860a admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:18.497741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e44b58b-2129-4464-b14f-dd9dcdae860a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:15:18.498053 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e44b58b-2129-4464-b14f-dd9dcdae860a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:18.500267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e44b58b-2129-4464-b14f-dd9dcdae860a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:18.521805 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1341/2681] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 20:15:18.521950 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396002d-3f50-4570-a688-82ff21b2c641 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:18.531985 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7396002d-3f50-4570-a688-82ff21b2c641 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:18.537336 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7b4ca-28a5-4d69-98d4-67690a7ee3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:18.538001 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7b4ca-28a5-4d69-98d4-67690a7ee3df admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:18.538391 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7b4ca-28a5-4d69-98d4-67690a7ee3df admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:18.538585 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7b4ca-28a5-4d69-98d4-67690a7ee3df admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:18.539225 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b7b4ca-28a5-4d69-98d4-67690a7ee3df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:18.541191 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b7b4ca-28a5-4d69-98d4-67690a7ee3df admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:18.674297 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1341/2682] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:15:18 2022] POST /identity/v3/projects => generated 361 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:15:18.678108 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44b58b-2129-4464-b14f-dd9dcdae860a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:18.691210 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44b58b-2129-4464-b14f-dd9dcdae860a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:18.760506 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5976f-592a-4f7a-925a-723509666ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:18.761490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5976f-592a-4f7a-925a-723509666ebf admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:18.761935 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5976f-592a-4f7a-925a-723509666ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:18.762386 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5976f-592a-4f7a-925a-723509666ebf admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:18.763245 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5976f-592a-4f7a-925a-723509666ebf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:18.765374 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5976f-592a-4f7a-925a-723509666ebf admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:18.862798 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1342/2683] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:15:18 2022] POST /identity/v3/users => generated 281 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:15:18.866777 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7b4ca-28a5-4d69-98d4-67690a7ee3df admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:18.879071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7b4ca-28a5-4d69-98d4-67690a7ee3df admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:18.884126 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136abe0-96e1-4df9-adfc-7d1af3b5346b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:18.884821 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136abe0-96e1-4df9-adfc-7d1af3b5346b admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:18.885150 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136abe0-96e1-4df9-adfc-7d1af3b5346b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:18.885426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136abe0-96e1-4df9-adfc-7d1af3b5346b admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:18.886144 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136abe0-96e1-4df9-adfc-7d1af3b5346b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:18.888647 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136abe0-96e1-4df9-adfc-7d1af3b5346b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:18.900520 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1342/2684] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:15:18.904272 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5976f-592a-4f7a-925a-723509666ebf admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:18.914900 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5976f-592a-4f7a-925a-723509666ebf admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:18.919793 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6e89f-9c5f-4011-aa1e-a6d1c0d9723b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:18.920493 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6e89f-9c5f-4011-aa1e-a6d1c0d9723b admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:18.920777 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6e89f-9c5f-4011-aa1e-a6d1c0d9723b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:18.921051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6e89f-9c5f-4011-aa1e-a6d1c0d9723b admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:18.921774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc6e89f-9c5f-4011-aa1e-a6d1c0d9723b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:18.923375 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc6e89f-9c5f-4011-aa1e-a6d1c0d9723b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:18.941510 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1343/2685] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:15:18.943161 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136abe0-96e1-4df9-adfc-7d1af3b5346b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:18.954986 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136abe0-96e1-4df9-adfc-7d1af3b5346b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:18.964638 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2a25a-4aa0-45ce-941d-9cd88579a3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:18.966002 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2a25a-4aa0-45ce-941d-9cd88579a3d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:18.966473 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2a25a-4aa0-45ce-941d-9cd88579a3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:18.966940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2a25a-4aa0-45ce-941d-9cd88579a3d2 admin admin] PATH_INFO: `/v3/projects/7474c9db83e24bbd9e47982c740012fd/users/bcbb8660d8d3430c84e4016422fb88af/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:18.987519 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d2a25a-4aa0-45ce-941d-9cd88579a3d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7474c9db83e24bbd9e47982c740012fd, user_id=bcbb8660d8d3430c84e4016422fb88af, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:18.990472 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d2a25a-4aa0-45ce-941d-9cd88579a3d2 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:19.087556 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1343/2686] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:15:18 2022] PUT /identity/v3/projects/7474c9db83e24bbd9e47982c740012fd/users/bcbb8660d8d3430c84e4016422fb88af/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:15:19.089188 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13f8a0-f16d-4e84-9b1c-a9fff3239402 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:19.089486 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13f8a0-f16d-4e84-9b1c-a9fff3239402 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:19.089768 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13f8a0-f16d-4e84-9b1c-a9fff3239402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:19.120404 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5b13f8a0-f16d-4e84-9b1c-a9fff3239402 None None] MFA Rules not processed for user `bcbb8660d8d3430c84e4016422fb88af`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:15:19.121633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5b13f8a0-f16d-4e84-9b1c-a9fff3239402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:19.198522 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1344/2687] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:15:19 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:15:19.230840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2a25a-4aa0-45ce-941d-9cd88579a3d2 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:19.233789 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-32d2a25a-4aa0-45ce-941d-9cd88579a3d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:19.242629 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:INSERT=4,keystone:DELETE=10,keystone:SELECT=86 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:15:19.306785 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2a25a-4aa0-45ce-941d-9cd88579a3d2 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:19.311108 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4d64d-74c2-4b11-96b9-c0982b622fab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:19.311762 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4d64d-74c2-4b11-96b9-c0982b622fab service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:19.312042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4d64d-74c2-4b11-96b9-c0982b622fab service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:19.312337 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4d64d-74c2-4b11-96b9-c0982b622fab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:19.322849 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e4d64d-74c2-4b11-96b9-c0982b622fab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:19.324291 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e4d64d-74c2-4b11-96b9-c0982b622fab service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:19.334538 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1344/2688] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:15:19 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:15:19.348873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13f8a0-f16d-4e84-9b1c-a9fff3239402 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:19.411790 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:15:19.424087 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13f8a0-f16d-4e84-9b1c-a9fff3239402 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:19.429459 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fa237-b80b-4373-8bdf-0ee1ee2aa947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:19.430453 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fa237-b80b-4373-8bdf-0ee1ee2aa947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:19.430739 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fa237-b80b-4373-8bdf-0ee1ee2aa947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:19.431011 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fa237-b80b-4373-8bdf-0ee1ee2aa947 admin admin] PATH_INFO: `/v3/users/bcbb8660d8d3430c84e4016422fb88af` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:19.432255 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9fa237-b80b-4373-8bdf-0ee1ee2aa947 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcbb8660d8d3430c84e4016422fb88af)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:19.434188 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9fa237-b80b-4373-8bdf-0ee1ee2aa947 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:19.577783 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-dc9fa237-b80b-4373-8bdf-0ee1ee2aa947 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcbb8660d8d3430c84e4016422fb88af'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:19.579192 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-dc9fa237-b80b-4373-8bdf-0ee1ee2aa947 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcbb8660d8d3430c84e4016422fb88af'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:19.593773 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-dc9fa237-b80b-4373-8bdf-0ee1ee2aa947 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcbb8660d8d3430c84e4016422fb88af'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:19.604163 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-dc9fa237-b80b-4373-8bdf-0ee1ee2aa947 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcbb8660d8d3430c84e4016422fb88af'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:20.037267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-dc9fa237-b80b-4373-8bdf-0ee1ee2aa947 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcbb8660d8d3430c84e4016422fb88af'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:20.044328 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1345/2689] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:15:19 2022] DELETE /identity/v3/users/bcbb8660d8d3430c84e4016422fb88af => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:15:20.050893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706b184-5c6d-471b-baf9-223eee27a354 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:20.050893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706b184-5c6d-471b-baf9-223eee27a354 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:20.050893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706b184-5c6d-471b-baf9-223eee27a354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:20.070353 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9706b184-5c6d-471b-baf9-223eee27a354 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:15:20.072344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9706b184-5c6d-471b-baf9-223eee27a354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:20.144744 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1345/2690] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:15:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:15:20.154396 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fa237-b80b-4373-8bdf-0ee1ee2aa947 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:20.155929 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-dc9fa237-b80b-4373-8bdf-0ee1ee2aa947 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:20.220539 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fa237-b80b-4373-8bdf-0ee1ee2aa947 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:20.224512 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970821e1-ae45-4ea6-8539-f51bbcd75557 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:20.225164 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970821e1-ae45-4ea6-8539-f51bbcd75557 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:20.225453 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970821e1-ae45-4ea6-8539-f51bbcd75557 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:20.236870 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970821e1-ae45-4ea6-8539-f51bbcd75557 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:20.250038 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970821e1-ae45-4ea6-8539-f51bbcd75557 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:20.251712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970821e1-ae45-4ea6-8539-f51bbcd75557 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:20.263848 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1346/2691] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:15:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:15:20.976269 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706b184-5c6d-471b-baf9-223eee27a354 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:20.977975 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9706b184-5c6d-471b-baf9-223eee27a354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:20.994116 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706b184-5c6d-471b-baf9-223eee27a354 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:20.999802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980081c1-be66-49f4-b15e-6dd33831c8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:21.000765 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980081c1-be66-49f4-b15e-6dd33831c8f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:21.000976 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980081c1-be66-49f4-b15e-6dd33831c8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:21.001247 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980081c1-be66-49f4-b15e-6dd33831c8f6 admin admin] PATH_INFO: `/v3/projects/7474c9db83e24bbd9e47982c740012fd` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:21.002508 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980081c1-be66-49f4-b15e-6dd33831c8f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7474c9db83e24bbd9e47982c740012fd)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:21.004482 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980081c1-be66-49f4-b15e-6dd33831c8f6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:21.132540 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-980081c1-be66-49f4-b15e-6dd33831c8f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7474c9db83e24bbd9e47982c740012fd', 'request_id': 'req-980081c1-be66-49f4-b15e-6dd33831c8f6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:21.235204 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-980081c1-be66-49f4-b15e-6dd33831c8f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7474c9db83e24bbd9e47982c740012fd', 'request_id': 'req-980081c1-be66-49f4-b15e-6dd33831c8f6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:21.243480 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-980081c1-be66-49f4-b15e-6dd33831c8f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7474c9db83e24bbd9e47982c740012fd', 'request_id': 'req-980081c1-be66-49f4-b15e-6dd33831c8f6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:21.259068 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-980081c1-be66-49f4-b15e-6dd33831c8f6 admin admin] The token cache is being invalidate because project 7474c9db83e24bbd9e47982c740012fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:15:21.259453 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-980081c1-be66-49f4-b15e-6dd33831c8f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:21.260179 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-980081c1-be66-49f4-b15e-6dd33831c8f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:15:21.262166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1346/2692] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:15:20 2022] DELETE /identity/v3/projects/7474c9db83e24bbd9e47982c740012fd => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:15:21.266876 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0ab54-da7d-4a0f-ad43-b8b7bdc349bd None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:21.267509 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0ab54-da7d-4a0f-ad43-b8b7bdc349bd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:21.267509 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0ab54-da7d-4a0f-ad43-b8b7bdc349bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:21.287002 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-27b0ab54-da7d-4a0f-ad43-b8b7bdc349bd None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:15:21.290865 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-27b0ab54-da7d-4a0f-ad43-b8b7bdc349bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:21.364116 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1347/2693] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:15:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:15:21.371978 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91116b-9196-4f8a-b110-fc6774c7c9b7 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:21.372271 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91116b-9196-4f8a-b110-fc6774c7c9b7 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:21.372585 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91116b-9196-4f8a-b110-fc6774c7c9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:21.395643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-aa91116b-9196-4f8a-b110-fc6774c7c9b7 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:15:21.396830 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-aa91116b-9196-4f8a-b110-fc6774c7c9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:21.408430 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1347/2694] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:15:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:15:21.410192 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b0ab54-da7d-4a0f-ad43-b8b7bdc349bd None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:21.420590 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b0ab54-da7d-4a0f-ad43-b8b7bdc349bd None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:21.425517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57ef73-acc2-4cc9-87b6-21b0594abc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:21.426394 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57ef73-acc2-4cc9-87b6-21b0594abc2d admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:21.426781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57ef73-acc2-4cc9-87b6-21b0594abc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:21.427152 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57ef73-acc2-4cc9-87b6-21b0594abc2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:21.427995 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57ef73-acc2-4cc9-87b6-21b0594abc2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:15:21.428450 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57ef73-acc2-4cc9-87b6-21b0594abc2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:21.430312 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57ef73-acc2-4cc9-87b6-21b0594abc2d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:21.443278 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1348/2695] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:15:21.448093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91116b-9196-4f8a-b110-fc6774c7c9b7 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:21.458692 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91116b-9196-4f8a-b110-fc6774c7c9b7 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:21.463901 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abbfb2b-9ec0-4c42-a4bf-ca3e13acc9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:21.464810 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbfb2b-9ec0-4c42-a4bf-ca3e13acc9fa admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:21.465720 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbfb2b-9ec0-4c42-a4bf-ca3e13acc9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:21.465720 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbfb2b-9ec0-4c42-a4bf-ca3e13acc9fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:21.466329 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abbfb2b-9ec0-4c42-a4bf-ca3e13acc9fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:21.468334 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abbfb2b-9ec0-4c42-a4bf-ca3e13acc9fa admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:21.620360 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1348/2696] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:15:21 2022] POST /identity/v3/projects => generated 365 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:15:21.625166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57ef73-acc2-4cc9-87b6-21b0594abc2d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:21.631306 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:15:21.637176 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57ef73-acc2-4cc9-87b6-21b0594abc2d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:21.698352 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae7d4c-05dd-4e4a-81d3-0b089de2cb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:21.698972 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae7d4c-05dd-4e4a-81d3-0b089de2cb31 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:21.699259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae7d4c-05dd-4e4a-81d3-0b089de2cb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:21.699538 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fae7d4c-05dd-4e4a-81d3-0b089de2cb31 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:21.700248 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae7d4c-05dd-4e4a-81d3-0b089de2cb31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:21.702273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fae7d4c-05dd-4e4a-81d3-0b089de2cb31 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:21.806529 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1349/2697] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:15:21 2022] POST /identity/v3/users => generated 283 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:15:21.807808 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abbfb2b-9ec0-4c42-a4bf-ca3e13acc9fa admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:21.813842 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:15:21.819582 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abbfb2b-9ec0-4c42-a4bf-ca3e13acc9fa admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:21.824545 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c4ca7-a856-4f4d-b85a-ac93939ca92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:21.825289 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c4ca7-a856-4f4d-b85a-ac93939ca92c admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:21.825682 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c4ca7-a856-4f4d-b85a-ac93939ca92c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:21.826060 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c4ca7-a856-4f4d-b85a-ac93939ca92c admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:21.828249 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c4ca7-a856-4f4d-b85a-ac93939ca92c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:21.830037 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c4ca7-a856-4f4d-b85a-ac93939ca92c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:21.847506 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1349/2698] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:15:21.854511 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae7d4c-05dd-4e4a-81d3-0b089de2cb31 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:21.867301 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fae7d4c-05dd-4e4a-81d3-0b089de2cb31 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:21.874089 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd581d-e475-4e5b-966d-16b493908d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:21.880513 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd581d-e475-4e5b-966d-16b493908d26 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:21.880513 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd581d-e475-4e5b-966d-16b493908d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:21.880513 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd581d-e475-4e5b-966d-16b493908d26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:21.880513 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd581d-e475-4e5b-966d-16b493908d26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:21.880513 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd581d-e475-4e5b-966d-16b493908d26 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:21.896123 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1350/2699] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:15:21.897399 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c4ca7-a856-4f4d-b85a-ac93939ca92c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:21.912084 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c4ca7-a856-4f4d-b85a-ac93939ca92c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:21.918443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b5e6b-08e6-463b-abb1-38450364f665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:21.919361 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b5e6b-08e6-463b-abb1-38450364f665 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:21.920460 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b5e6b-08e6-463b-abb1-38450364f665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:21.920460 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b5e6b-08e6-463b-abb1-38450364f665 admin admin] PATH_INFO: `/v3/projects/10ccf22c04034cd19c387fa9122b8235/users/1cd1dd8949da485a87dafc94587cc5ad/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:21.937634 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082b5e6b-08e6-463b-abb1-38450364f665 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10ccf22c04034cd19c387fa9122b8235, user_id=1cd1dd8949da485a87dafc94587cc5ad, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:21.938688 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082b5e6b-08e6-463b-abb1-38450364f665 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:22.025755 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1350/2700] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:15:21 2022] PUT /identity/v3/projects/10ccf22c04034cd19c387fa9122b8235/users/1cd1dd8949da485a87dafc94587cc5ad/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:15:22.028657 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a547b-25e0-4b8c-bea0-b6eba3922258 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:22.028956 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a547b-25e0-4b8c-bea0-b6eba3922258 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:22.029240 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a547b-25e0-4b8c-bea0-b6eba3922258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:22.063715 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ba6a547b-25e0-4b8c-bea0-b6eba3922258 None None] MFA Rules not processed for user `1cd1dd8949da485a87dafc94587cc5ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:15:22.065003 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ba6a547b-25e0-4b8c-bea0-b6eba3922258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:22.146124 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1351/2701] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:15:22 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:15:22.157543 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b5e6b-08e6-463b-abb1-38450364f665 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:22.159304 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-082b5e6b-08e6-463b-abb1-38450364f665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:22.230905 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b5e6b-08e6-463b-abb1-38450364f665 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:22.240881 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36862327-b397-4a0b-96b4-b11836f4ae97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:22.241483 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36862327-b397-4a0b-96b4-b11836f4ae97 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:22.241769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36862327-b397-4a0b-96b4-b11836f4ae97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:22.242044 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36862327-b397-4a0b-96b4-b11836f4ae97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:22.253248 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36862327-b397-4a0b-96b4-b11836f4ae97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:22.255346 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36862327-b397-4a0b-96b4-b11836f4ae97 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:22.265515 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1351/2702] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:15:22 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:15:25.164967 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21211b-b444-4921-990b-637396b77225 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:25.165448 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21211b-b444-4921-990b-637396b77225 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:25.165547 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21211b-b444-4921-990b-637396b77225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:25.184213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8a21211b-b444-4921-990b-637396b77225 None None] MFA Rules not processed for user `a9fe5380b3974516ac4e5b2fd9284348`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:15:25.185469 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8a21211b-b444-4921-990b-637396b77225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:25.242429 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1352/2703] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:15:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:15:31.463114 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36862327-b397-4a0b-96b4-b11836f4ae97 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:31.464569 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-36862327-b397-4a0b-96b4-b11836f4ae97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:31.533042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36862327-b397-4a0b-96b4-b11836f4ae97 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:31.538566 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0c0db-a5aa-4410-a7ae-5c6710c5ef2d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9715d5deeccd419b97cb0bd39e17982f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:31.539199 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0c0db-a5aa-4410-a7ae-5c6710c5ef2d service placement] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:31.539475 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0c0db-a5aa-4410-a7ae-5c6710c5ef2d service placement] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:31.539742 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0c0db-a5aa-4410-a7ae-5c6710c5ef2d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:31.541482 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4ba0c0db-a5aa-4410-a7ae-5c6710c5ef2d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:31.553184 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0c0db-a5aa-4410-a7ae-5c6710c5ef2d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:31.554898 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0c0db-a5aa-4410-a7ae-5c6710c5ef2d service placement] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:31.564661 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1352/2704] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:15:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:15:35.233681 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:15:37.396682 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88175494-2bdc-40a0-8fe6-aa312d200e39 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:37.397275 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88175494-2bdc-40a0-8fe6-aa312d200e39 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:37.397275 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88175494-2bdc-40a0-8fe6-aa312d200e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:37.416117 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-88175494-2bdc-40a0-8fe6-aa312d200e39 None None] MFA Rules not processed for user `1d37f369749143ce813c693adc5f2ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:15:37.417364 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-88175494-2bdc-40a0-8fe6-aa312d200e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:37.481575 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1353/2705] 10.222.0.44 () {58 vars in 1009 bytes} [Thu Jun 23 20:15:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:15:37.496143 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0c0db-a5aa-4410-a7ae-5c6710c5ef2d service placement] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:37.498051 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4ba0c0db-a5aa-4410-a7ae-5c6710c5ef2d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:37.565105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:15:37.571230 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0c0db-a5aa-4410-a7ae-5c6710c5ef2d service placement] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:37.577293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3570f-a38c-46ea-bb66-9b39bee44867 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96db03079816461bb322633f64b9fb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:37.577946 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3570f-a38c-46ea-bb66-9b39bee44867 service swift] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:37.578226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3570f-a38c-46ea-bb66-9b39bee44867 service swift] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:37.578497 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3570f-a38c-46ea-bb66-9b39bee44867 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:37.587916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3570f-a38c-46ea-bb66-9b39bee44867 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:37.589694 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3570f-a38c-46ea-bb66-9b39bee44867 service swift] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:37.598693 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1353/2706] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jun 23 20:15:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 20:15:47.473704 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=25 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:15:47.802547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:15:51.351312 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caaed90-dab1-4877-9231-c70add3ac0c1 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:51.351915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caaed90-dab1-4877-9231-c70add3ac0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:51.351915 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caaed90-dab1-4877-9231-c70add3ac0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:51.369126 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8caaed90-dab1-4877-9231-c70add3ac0c1 None None] MFA Rules not processed for user `2c5e6ea89435497c8b1b06f42e08c82e`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:15:51.370318 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8caaed90-dab1-4877-9231-c70add3ac0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:51.424869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1354/2707] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:15:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:15:58.037848 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3570f-a38c-46ea-bb66-9b39bee44867 service swift] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:15:58.040070 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-23c3570f-a38c-46ea-bb66-9b39bee44867 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:58.153105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3570f-a38c-46ea-bb66-9b39bee44867 service swift] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:15:58.159945 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8ebcc9-c86b-4817-8292-fc5559f80171 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:15:58.160763 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ebcc9-c86b-4817-8292-fc5559f80171 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:15:58.161045 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ebcc9-c86b-4817-8292-fc5559f80171 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:15:58.161344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8ebcc9-c86b-4817-8292-fc5559f80171 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:15:58.163138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9a8ebcc9-c86b-4817-8292-fc5559f80171 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:15:58.175738 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8ebcc9-c86b-4817-8292-fc5559f80171 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:15:58.177414 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8ebcc9-c86b-4817-8292-fc5559f80171 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:15:58.197362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1354/2708] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:15:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:16:01.418277 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=25 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:16:02.543778 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caaed90-dab1-4877-9231-c70add3ac0c1 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:16:02.605255 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caaed90-dab1-4877-9231-c70add3ac0c1 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:16:02.609958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb99354-29f8-44e6-a4cc-65156e0ac5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:16:02.610885 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb99354-29f8-44e6-a4cc-65156e0ac5d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:16:02.611163 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb99354-29f8-44e6-a4cc-65156e0ac5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:16:02.611437 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb99354-29f8-44e6-a4cc-65156e0ac5d5 admin admin] PATH_INFO: `/v3/users/1cd1dd8949da485a87dafc94587cc5ad` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:16:02.612659 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb99354-29f8-44e6-a4cc-65156e0ac5d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cd1dd8949da485a87dafc94587cc5ad)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:16:02.615211 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb99354-29f8-44e6-a4cc-65156e0ac5d5 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:16:02.825988 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8eb99354-29f8-44e6-a4cc-65156e0ac5d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cd1dd8949da485a87dafc94587cc5ad'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:02.826640 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8eb99354-29f8-44e6-a4cc-65156e0ac5d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cd1dd8949da485a87dafc94587cc5ad'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:02.839236 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8eb99354-29f8-44e6-a4cc-65156e0ac5d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cd1dd8949da485a87dafc94587cc5ad'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:02.846114 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8eb99354-29f8-44e6-a4cc-65156e0ac5d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cd1dd8949da485a87dafc94587cc5ad'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:02.915509 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8eb99354-29f8-44e6-a4cc-65156e0ac5d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cd1dd8949da485a87dafc94587cc5ad'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:02.922881 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1355/2709] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:16:02 2022] DELETE /identity/v3/users/1cd1dd8949da485a87dafc94587cc5ad => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:16:02.927507 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37666174-e3da-4c94-a9c3-6c5cfb3cfce3 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:16:02.927955 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37666174-e3da-4c94-a9c3-6c5cfb3cfce3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:16:02.928052 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37666174-e3da-4c94-a9c3-6c5cfb3cfce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:16:02.945896 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-37666174-e3da-4c94-a9c3-6c5cfb3cfce3 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:16:02.947125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-37666174-e3da-4c94-a9c3-6c5cfb3cfce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:16:03.008268 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1355/2710] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:16:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:16:03.017618 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb99354-29f8-44e6-a4cc-65156e0ac5d5 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:16:03.019091 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8eb99354-29f8-44e6-a4cc-65156e0ac5d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:16:03.093256 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb99354-29f8-44e6-a4cc-65156e0ac5d5 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:16:03.096855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5156bf-a352-4e2d-a6b8-957df0f9d36c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:16:03.097565 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5156bf-a352-4e2d-a6b8-957df0f9d36c service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:16:03.097847 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5156bf-a352-4e2d-a6b8-957df0f9d36c service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:16:03.098147 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5156bf-a352-4e2d-a6b8-957df0f9d36c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:16:03.107474 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5156bf-a352-4e2d-a6b8-957df0f9d36c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:16:03.108958 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5156bf-a352-4e2d-a6b8-957df0f9d36c service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:16:03.119035 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1356/2711] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:16:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:16:03.517866 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37666174-e3da-4c94-a9c3-6c5cfb3cfce3 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:16:03.523419 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-37666174-e3da-4c94-a9c3-6c5cfb3cfce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:16:03.533851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37666174-e3da-4c94-a9c3-6c5cfb3cfce3 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:16:03.540844 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56454734-0e55-4c2e-8e5e-3a719b9d95f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:16:03.541856 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56454734-0e55-4c2e-8e5e-3a719b9d95f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:16:03.542201 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56454734-0e55-4c2e-8e5e-3a719b9d95f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:16:03.542555 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56454734-0e55-4c2e-8e5e-3a719b9d95f8 admin admin] PATH_INFO: `/v3/projects/10ccf22c04034cd19c387fa9122b8235` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:16:03.544000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56454734-0e55-4c2e-8e5e-3a719b9d95f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10ccf22c04034cd19c387fa9122b8235)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:16:03.546456 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56454734-0e55-4c2e-8e5e-3a719b9d95f8 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:16:03.658443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-56454734-0e55-4c2e-8e5e-3a719b9d95f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10ccf22c04034cd19c387fa9122b8235', 'request_id': 'req-56454734-0e55-4c2e-8e5e-3a719b9d95f8'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:03.732272 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-56454734-0e55-4c2e-8e5e-3a719b9d95f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10ccf22c04034cd19c387fa9122b8235', 'request_id': 'req-56454734-0e55-4c2e-8e5e-3a719b9d95f8'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:03.739046 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-56454734-0e55-4c2e-8e5e-3a719b9d95f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10ccf22c04034cd19c387fa9122b8235', 'request_id': 'req-56454734-0e55-4c2e-8e5e-3a719b9d95f8'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:03.751387 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-56454734-0e55-4c2e-8e5e-3a719b9d95f8 admin admin] The token cache is being invalidate because project 10ccf22c04034cd19c387fa9122b8235 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:16:03.751792 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-56454734-0e55-4c2e-8e5e-3a719b9d95f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:03.752522 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-56454734-0e55-4c2e-8e5e-3a719b9d95f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:03.754154 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1356/2712] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:16:03 2022] DELETE /identity/v3/projects/10ccf22c04034cd19c387fa9122b8235 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:16:03.759074 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f2557-36f1-4254-b0cb-2a812003faa6 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:16:03.759564 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f2557-36f1-4254-b0cb-2a812003faa6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:16:03.759673 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f2557-36f1-4254-b0cb-2a812003faa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:16:03.779308 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-117f2557-36f1-4254-b0cb-2a812003faa6 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:16:03.780619 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-117f2557-36f1-4254-b0cb-2a812003faa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:16:03.799113 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:16:03.842603 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1357/2713] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:16:03.850876 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81904b-3cef-4410-bab1-6e6706f5e9c6 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:16:03.851166 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81904b-3cef-4410-bab1-6e6706f5e9c6 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:16:03.851475 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81904b-3cef-4410-bab1-6e6706f5e9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:16:03.870731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-0c81904b-3cef-4410-bab1-6e6706f5e9c6 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:16:03.871900 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0c81904b-3cef-4410-bab1-6e6706f5e9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:16:03.879869 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1357/2714] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:16:03.884060 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f2557-36f1-4254-b0cb-2a812003faa6 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:16:03.893633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f2557-36f1-4254-b0cb-2a812003faa6 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:16:03.898725 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e156d-2f5c-4c2a-acd7-36aed54e901f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:16:03.899371 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e156d-2f5c-4c2a-acd7-36aed54e901f admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:16:03.899655 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e156d-2f5c-4c2a-acd7-36aed54e901f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:16:03.899935 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e156d-2f5c-4c2a-acd7-36aed54e901f admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:16:03.900687 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e156d-2f5c-4c2a-acd7-36aed54e901f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:16:03.900991 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e156d-2f5c-4c2a-acd7-36aed54e901f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:16:03.902580 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29e156d-2f5c-4c2a-acd7-36aed54e901f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:16:03.914866 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1358/2715] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:16:03.918952 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81904b-3cef-4410-bab1-6e6706f5e9c6 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:16:03.923406 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:16:03.928581 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81904b-3cef-4410-bab1-6e6706f5e9c6 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:16:03.933210 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51810899-6e1c-4f56-be77-6d73a3892a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:16:03.933869 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51810899-6e1c-4f56-be77-6d73a3892a8d admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:16:03.934964 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51810899-6e1c-4f56-be77-6d73a3892a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:16:03.934964 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51810899-6e1c-4f56-be77-6d73a3892a8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:16:03.935137 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51810899-6e1c-4f56-be77-6d73a3892a8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:16:03.937186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51810899-6e1c-4f56-be77-6d73a3892a8d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:16:04.014294 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1358/2716] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:16:03 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:16:04.018488 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e156d-2f5c-4c2a-acd7-36aed54e901f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:16:04.029276 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29e156d-2f5c-4c2a-acd7-36aed54e901f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:16:04.089233 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e5a40-a67d-4677-945c-0c84998d9728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:16:04.089907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e5a40-a67d-4677-945c-0c84998d9728 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:16:04.090213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e5a40-a67d-4677-945c-0c84998d9728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:16:04.090486 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e5a40-a67d-4677-945c-0c84998d9728 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:16:04.091183 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e5a40-a67d-4677-945c-0c84998d9728 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:16:04.093157 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e5a40-a67d-4677-945c-0c84998d9728 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:16:04.173242 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1359/2717] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:16:04 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:16:04.177309 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51810899-6e1c-4f56-be77-6d73a3892a8d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:16:04.187647 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51810899-6e1c-4f56-be77-6d73a3892a8d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:16:04.192736 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71f81b-8750-4e0e-84d5-51ce95ebd5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:16:04.193595 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71f81b-8750-4e0e-84d5-51ce95ebd5bf admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:16:04.194400 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71f81b-8750-4e0e-84d5-51ce95ebd5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:16:04.194672 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71f81b-8750-4e0e-84d5-51ce95ebd5bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:16:04.195395 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c71f81b-8750-4e0e-84d5-51ce95ebd5bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:16:04.197065 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c71f81b-8750-4e0e-84d5-51ce95ebd5bf admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:16:04.207379 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1359/2718] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:16:04.211813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e5a40-a67d-4677-945c-0c84998d9728 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:16:04.221787 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e5a40-a67d-4677-945c-0c84998d9728 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:16:04.232646 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f911708-3c69-42f0-8d50-9e5575811290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:16:04.233332 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f911708-3c69-42f0-8d50-9e5575811290 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:16:04.233613 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f911708-3c69-42f0-8d50-9e5575811290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:16:04.233885 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f911708-3c69-42f0-8d50-9e5575811290 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:16:04.234619 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f911708-3c69-42f0-8d50-9e5575811290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:16:04.236229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f911708-3c69-42f0-8d50-9e5575811290 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:16:04.249177 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1360/2719] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:16:04.253347 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71f81b-8750-4e0e-84d5-51ce95ebd5bf admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:16:04.263211 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71f81b-8750-4e0e-84d5-51ce95ebd5bf admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:16:04.267731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b2051-e120-40e1-ab80-0e5a32903f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:16:04.268627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b2051-e120-40e1-ab80-0e5a32903f7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:16:04.268907 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b2051-e120-40e1-ab80-0e5a32903f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:16:04.269175 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b2051-e120-40e1-ab80-0e5a32903f7a admin admin] PATH_INFO: `/v3/projects/eccb032b11e045fdbb42238762eada05/users/1b02048ea85d46889e86bb8d073bf456/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:16:04.283131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9b2051-e120-40e1-ab80-0e5a32903f7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eccb032b11e045fdbb42238762eada05, user_id=1b02048ea85d46889e86bb8d073bf456, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:16:04.285803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9b2051-e120-40e1-ab80-0e5a32903f7a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:16:04.366366 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1360/2720] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:16:04 2022] PUT /identity/v3/projects/eccb032b11e045fdbb42238762eada05/users/1b02048ea85d46889e86bb8d073bf456/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:16:04.371410 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b9801-6e89-45c7-83e4-20d652465b0f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:16:04.371703 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b9801-6e89-45c7-83e4-20d652465b0f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:16:04.371980 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b9801-6e89-45c7-83e4-20d652465b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:16:04.517402 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-934b9801-6e89-45c7-83e4-20d652465b0f None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:16:04.518667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-934b9801-6e89-45c7-83e4-20d652465b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:16:04.577524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1361/2721] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:16:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:16:04.587803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b2051-e120-40e1-ab80-0e5a32903f7a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:16:04.589500 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ec9b2051-e120-40e1-ab80-0e5a32903f7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:16:04.643847 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b2051-e120-40e1-ab80-0e5a32903f7a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:16:04.647865 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf662d3-cc0e-4f91-8de0-3766a20952c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:16:04.648550 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf662d3-cc0e-4f91-8de0-3766a20952c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:16:04.648826 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf662d3-cc0e-4f91-8de0-3766a20952c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:16:04.649094 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf662d3-cc0e-4f91-8de0-3766a20952c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:16:04.657917 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf662d3-cc0e-4f91-8de0-3766a20952c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:16:04.659332 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf662d3-cc0e-4f91-8de0-3766a20952c2 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:16:04.669103 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1361/2722] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:16:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:16:05.985134 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe25151-bfd5-4a74-bef7-dbe385b49af1 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:16:05.985989 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe25151-bfd5-4a74-bef7-dbe385b49af1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:16:05.986267 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe25151-bfd5-4a74-bef7-dbe385b49af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:16:06.018128 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5fe25151-bfd5-4a74-bef7-dbe385b49af1 None None] MFA Rules not processed for user `1b02048ea85d46889e86bb8d073bf456`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:16:06.019765 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5fe25151-bfd5-4a74-bef7-dbe385b49af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:16:06.039315 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:16:06.098259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1362/2723] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:16:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:16:06.109266 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf662d3-cc0e-4f91-8de0-3766a20952c2 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:16:06.111293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-faf662d3-cc0e-4f91-8de0-3766a20952c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:16:06.171321 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf662d3-cc0e-4f91-8de0-3766a20952c2 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:16:06.175369 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383079bb-8fe3-4a37-894c-af7ff7773d09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:16:06.176350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383079bb-8fe3-4a37-894c-af7ff7773d09 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:16:06.176494 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383079bb-8fe3-4a37-894c-af7ff7773d09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:16:06.176799 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383079bb-8fe3-4a37-894c-af7ff7773d09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:16:06.187314 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383079bb-8fe3-4a37-894c-af7ff7773d09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:16:06.188826 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383079bb-8fe3-4a37-894c-af7ff7773d09 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:16:06.208576 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1362/2724] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:16:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:16:07.955864 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf38908-3bfe-462e-850a-88ae4b902e6a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:16:07.956474 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf38908-3bfe-462e-850a-88ae4b902e6a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:16:07.956979 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf38908-3bfe-462e-850a-88ae4b902e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:16:07.974760 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-edf38908-3bfe-462e-850a-88ae4b902e6a None None] MFA Rules not processed for user `1b02048ea85d46889e86bb8d073bf456`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:16:07.976538 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-edf38908-3bfe-462e-850a-88ae4b902e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:16:07.984454 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1363/2725] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:16:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:16:13.466536 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383079bb-8fe3-4a37-894c-af7ff7773d09 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:16:13.476600 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383079bb-8fe3-4a37-894c-af7ff7773d09 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:16:13.480765 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93891e-3d80-4cde-aaf7-6f1459ade624 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:16:13.481592 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93891e-3d80-4cde-aaf7-6f1459ade624 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:16:13.481902 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93891e-3d80-4cde-aaf7-6f1459ade624 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:16:13.482174 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93891e-3d80-4cde-aaf7-6f1459ade624 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:16:13.483934 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6a93891e-3d80-4cde-aaf7-6f1459ade624 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:16:13.531553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:16:13.546443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93891e-3d80-4cde-aaf7-6f1459ade624 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:16:13.547886 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93891e-3d80-4cde-aaf7-6f1459ade624 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:16:13.557992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1363/2726] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:16:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:16:17.969429 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=22 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:16:20.887664 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf38908-3bfe-462e-850a-88ae4b902e6a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:16:20.889089 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-edf38908-3bfe-462e-850a-88ae4b902e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:16:20.910413 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf38908-3bfe-462e-850a-88ae4b902e6a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:16:20.910413 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54713df9-3be4-4d31-ace2-45d44ce6d5c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:16:20.910413 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54713df9-3be4-4d31-ace2-45d44ce6d5c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:16:20.910413 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54713df9-3be4-4d31-ace2-45d44ce6d5c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:16:20.910867 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54713df9-3be4-4d31-ace2-45d44ce6d5c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:16:20.910867 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-54713df9-3be4-4d31-ace2-45d44ce6d5c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:16:20.919120 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54713df9-3be4-4d31-ace2-45d44ce6d5c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:16:20.920605 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54713df9-3be4-4d31-ace2-45d44ce6d5c3 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:16:20.931484 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1364/2727] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:16:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:16:23.601156 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=7 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:16:28.195845 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93891e-3d80-4cde-aaf7-6f1459ade624 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:16:28.197369 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6a93891e-3d80-4cde-aaf7-6f1459ade624 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:16:28.283099 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93891e-3d80-4cde-aaf7-6f1459ade624 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:16:28.285384 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caaf947-cdba-4d03-a708-b662886a71c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9715d5deeccd419b97cb0bd39e17982f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:16:28.286038 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caaf947-cdba-4d03-a708-b662886a71c0 service placement] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:16:28.286321 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caaf947-cdba-4d03-a708-b662886a71c0 service placement] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:16:28.286595 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caaf947-cdba-4d03-a708-b662886a71c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:16:28.288966 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9caaf947-cdba-4d03-a708-b662886a71c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:16:28.301008 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caaf947-cdba-4d03-a708-b662886a71c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:16:28.302633 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caaf947-cdba-4d03-a708-b662886a71c0 service placement] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:16:28.314781 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1364/2728] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:16:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:16:28.321377 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54713df9-3be4-4d31-ace2-45d44ce6d5c3 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:16:28.321377 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-54713df9-3be4-4d31-ace2-45d44ce6d5c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:16:28.338117 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54713df9-3be4-4d31-ace2-45d44ce6d5c3 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:16:28.344192 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3030a2-ec7c-458d-8be0-cc410ea2267b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:16:28.345683 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3030a2-ec7c-458d-8be0-cc410ea2267b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:16:28.346208 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3030a2-ec7c-458d-8be0-cc410ea2267b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:16:28.346856 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3030a2-ec7c-458d-8be0-cc410ea2267b admin admin] PATH_INFO: `/v3/users/eafea4329d6243459dd3a6add1e91a2e` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:16:28.348518 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3030a2-ec7c-458d-8be0-cc410ea2267b admin admin] RBAC: Authorizing `identity:delete_user(user_id=eafea4329d6243459dd3a6add1e91a2e)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:16:28.351684 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3030a2-ec7c-458d-8be0-cc410ea2267b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:16:28.546579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-bf3030a2-ec7c-458d-8be0-cc410ea2267b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eafea4329d6243459dd3a6add1e91a2e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:28.547155 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-bf3030a2-ec7c-458d-8be0-cc410ea2267b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eafea4329d6243459dd3a6add1e91a2e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:28.559709 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-bf3030a2-ec7c-458d-8be0-cc410ea2267b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eafea4329d6243459dd3a6add1e91a2e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:28.567624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-bf3030a2-ec7c-458d-8be0-cc410ea2267b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eafea4329d6243459dd3a6add1e91a2e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:28.658504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-bf3030a2-ec7c-458d-8be0-cc410ea2267b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eafea4329d6243459dd3a6add1e91a2e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:28.666555 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1365/2729] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:16:28 2022] DELETE /identity/v3/users/eafea4329d6243459dd3a6add1e91a2e => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:16:28.906218 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caaf947-cdba-4d03-a708-b662886a71c0 service placement] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:16:28.917343 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caaf947-cdba-4d03-a708-b662886a71c0 service placement] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:16:28.978182 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cbe6f4-7bb2-437a-acb7-6d93739c898a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:16:28.979049 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbe6f4-7bb2-437a-acb7-6d93739c898a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:16:28.979335 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbe6f4-7bb2-437a-acb7-6d93739c898a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:16:28.979609 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cbe6f4-7bb2-437a-acb7-6d93739c898a admin admin] PATH_INFO: `/v3/projects/edb569a8b39a437ea8be940df098268c` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:16:28.980863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cbe6f4-7bb2-437a-acb7-6d93739c898a admin admin] RBAC: Authorizing `identity:delete_project(project_id=edb569a8b39a437ea8be940df098268c)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:16:28.982799 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cbe6f4-7bb2-437a-acb7-6d93739c898a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:16:29.109870 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f5cbe6f4-7bb2-437a-acb7-6d93739c898a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edb569a8b39a437ea8be940df098268c', 'request_id': 'req-f5cbe6f4-7bb2-437a-acb7-6d93739c898a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:29.178498 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f5cbe6f4-7bb2-437a-acb7-6d93739c898a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edb569a8b39a437ea8be940df098268c', 'request_id': 'req-f5cbe6f4-7bb2-437a-acb7-6d93739c898a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:29.185289 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f5cbe6f4-7bb2-437a-acb7-6d93739c898a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edb569a8b39a437ea8be940df098268c', 'request_id': 'req-f5cbe6f4-7bb2-437a-acb7-6d93739c898a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:29.197798 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f5cbe6f4-7bb2-437a-acb7-6d93739c898a admin admin] The token cache is being invalidate because project edb569a8b39a437ea8be940df098268c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:16:29.198464 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f5cbe6f4-7bb2-437a-acb7-6d93739c898a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:29.199405 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f5cbe6f4-7bb2-437a-acb7-6d93739c898a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:29.200383 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1365/2730] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:16:28 2022] DELETE /identity/v3/projects/edb569a8b39a437ea8be940df098268c => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:16:29.204837 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3030a2-ec7c-458d-8be0-cc410ea2267b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:16:29.206311 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-bf3030a2-ec7c-458d-8be0-cc410ea2267b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:16:29.220462 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3030a2-ec7c-458d-8be0-cc410ea2267b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:16:29.277554 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b81a06-808a-4a92-af6d-4f4018fff47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:16:29.278429 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b81a06-808a-4a92-af6d-4f4018fff47a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:16:29.278712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b81a06-808a-4a92-af6d-4f4018fff47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:16:29.278987 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b81a06-808a-4a92-af6d-4f4018fff47a admin admin] PATH_INFO: `/v3/users/c04d712cc71545ffad09b88a1392c5c7` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:16:29.280146 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b81a06-808a-4a92-af6d-4f4018fff47a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c04d712cc71545ffad09b88a1392c5c7)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:16:29.282159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b81a06-808a-4a92-af6d-4f4018fff47a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:16:29.446779 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c1b81a06-808a-4a92-af6d-4f4018fff47a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c04d712cc71545ffad09b88a1392c5c7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:29.447463 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c1b81a06-808a-4a92-af6d-4f4018fff47a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c04d712cc71545ffad09b88a1392c5c7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:29.458560 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:16:29.461059 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c1b81a06-808a-4a92-af6d-4f4018fff47a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c04d712cc71545ffad09b88a1392c5c7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:29.467771 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c1b81a06-808a-4a92-af6d-4f4018fff47a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c04d712cc71545ffad09b88a1392c5c7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:29.526985 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c1b81a06-808a-4a92-af6d-4f4018fff47a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c04d712cc71545ffad09b88a1392c5c7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:29.533695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1366/2731] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:16:29 2022] DELETE /identity/v3/users/c04d712cc71545ffad09b88a1392c5c7 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:16:29.756979 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cbe6f4-7bb2-437a-acb7-6d93739c898a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:16:29.762351 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f5cbe6f4-7bb2-437a-acb7-6d93739c898a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:16:29.774631 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cbe6f4-7bb2-437a-acb7-6d93739c898a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:16:29.801816 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:16:29.844128 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1cc417-0a40-424e-9312-b6582681e52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:16:29.845027 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cc417-0a40-424e-9312-b6582681e52f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:16:29.845355 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cc417-0a40-424e-9312-b6582681e52f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:16:29.845631 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1cc417-0a40-424e-9312-b6582681e52f admin admin] PATH_INFO: `/v3/projects/95ab1ee032d74c4fbc1c544a09ffea80` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:16:29.846828 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1cc417-0a40-424e-9312-b6582681e52f admin admin] RBAC: Authorizing `identity:delete_project(project_id=95ab1ee032d74c4fbc1c544a09ffea80)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:16:29.848839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1cc417-0a40-424e-9312-b6582681e52f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:16:29.933823 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9e1cc417-0a40-424e-9312-b6582681e52f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95ab1ee032d74c4fbc1c544a09ffea80', 'request_id': 'req-9e1cc417-0a40-424e-9312-b6582681e52f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:29.972621 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9e1cc417-0a40-424e-9312-b6582681e52f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95ab1ee032d74c4fbc1c544a09ffea80', 'request_id': 'req-9e1cc417-0a40-424e-9312-b6582681e52f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:29.979531 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9e1cc417-0a40-424e-9312-b6582681e52f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95ab1ee032d74c4fbc1c544a09ffea80', 'request_id': 'req-9e1cc417-0a40-424e-9312-b6582681e52f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:29.991416 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9e1cc417-0a40-424e-9312-b6582681e52f admin admin] The token cache is being invalidate because project 95ab1ee032d74c4fbc1c544a09ffea80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:16:29.991699 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9e1cc417-0a40-424e-9312-b6582681e52f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:29.992377 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9e1cc417-0a40-424e-9312-b6582681e52f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:16:29.993953 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1366/2732] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:16:29 2022] DELETE /identity/v3/projects/95ab1ee032d74c4fbc1c544a09ffea80 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:16:39.561815 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:DELETE=2,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:16:39.987843 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:17:17.752135 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b81a06-808a-4a92-af6d-4f4018fff47a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:17.753594 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c1b81a06-808a-4a92-af6d-4f4018fff47a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:17.813589 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b81a06-808a-4a92-af6d-4f4018fff47a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:17.862229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d50451-a0bb-4951-bf5b-84345500521e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:17.862868 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d50451-a0bb-4951-bf5b-84345500521e service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:17.863147 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d50451-a0bb-4951-bf5b-84345500521e service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:17.863416 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d50451-a0bb-4951-bf5b-84345500521e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:17.865184 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-93d50451-a0bb-4951-bf5b-84345500521e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:17.924782 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d50451-a0bb-4951-bf5b-84345500521e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:17.926229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d50451-a0bb-4951-bf5b-84345500521e service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:17.935416 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1367/2733] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:17:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:17:21.131026 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86aafb-0827-48ce-8e1b-44a8f7bef123 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:21.131529 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86aafb-0827-48ce-8e1b-44a8f7bef123 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:21.131633 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86aafb-0827-48ce-8e1b-44a8f7bef123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:21.148841 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-6c86aafb-0827-48ce-8e1b-44a8f7bef123 None None] MFA Rules not processed for user `1b02048ea85d46889e86bb8d073bf456`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:17:21.150056 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6c86aafb-0827-48ce-8e1b-44a8f7bef123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:21.203432 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1367/2734] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:17:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:17:21.238950 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d50451-a0bb-4951-bf5b-84345500521e service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:21.239908 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-93d50451-a0bb-4951-bf5b-84345500521e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:21.296813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d50451-a0bb-4951-bf5b-84345500521e service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:21.300404 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb680b-3813-4a8b-ae63-1180f703fb8b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1dd72f952df34ad8853fe84673a41f45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:21.301044 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb680b-3813-4a8b-ae63-1180f703fb8b service glance] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:21.301322 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb680b-3813-4a8b-ae63-1180f703fb8b service glance] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:21.301595 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb680b-3813-4a8b-ae63-1180f703fb8b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:21.303439 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1bdb680b-3813-4a8b-ae63-1180f703fb8b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:21.328100 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:17:21.372190 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdb680b-3813-4a8b-ae63-1180f703fb8b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:21.373691 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdb680b-3813-4a8b-ae63-1180f703fb8b service glance] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:21.384367 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1368/2735] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:17:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 20:17:21.986727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86aafb-0827-48ce-8e1b-44a8f7bef123 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:21.987981 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6c86aafb-0827-48ce-8e1b-44a8f7bef123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:22.000612 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb680b-3813-4a8b-ae63-1180f703fb8b service glance] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:22.002195 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1bdb680b-3813-4a8b-ae63-1180f703fb8b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:22.063813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb680b-3813-4a8b-ae63-1180f703fb8b service glance] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:22.064122 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86aafb-0827-48ce-8e1b-44a8f7bef123 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:22.068889 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9092b-5f64-4113-9145-f3aa4d66ede6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:22.069462 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0bb887-c5da-4f00-a087-7d7a7e431214 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9715d5deeccd419b97cb0bd39e17982f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:22.069640 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9092b-5f64-4113-9145-f3aa4d66ede6 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:22.069873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9092b-5f64-4113-9145-f3aa4d66ede6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:22.070152 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9092b-5f64-4113-9145-f3aa4d66ede6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:22.070283 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0bb887-c5da-4f00-a087-7d7a7e431214 service placement] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:22.070658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0bb887-c5da-4f00-a087-7d7a7e431214 service placement] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:22.070932 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0bb887-c5da-4f00-a087-7d7a7e431214 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:22.072641 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-bcd9092b-5f64-4113-9145-f3aa4d66ede6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:22.072790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2e0bb887-c5da-4f00-a087-7d7a7e431214 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:22.084529 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0bb887-c5da-4f00-a087-7d7a7e431214 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:22.086426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0bb887-c5da-4f00-a087-7d7a7e431214 service placement] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:22.087875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9092b-5f64-4113-9145-f3aa4d66ede6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:22.089953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9092b-5f64-4113-9145-f3aa4d66ede6 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:22.097365 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1368/2736] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:17:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:17:22.101270 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1369/2737] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:17:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:17:32.093350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=57 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:17:32.096117 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=48 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:17:34.626103 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0bb887-c5da-4f00-a087-7d7a7e431214 service placement] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:34.627531 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2e0bb887-c5da-4f00-a087-7d7a7e431214 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:34.639658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0bb887-c5da-4f00-a087-7d7a7e431214 service placement] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:34.643389 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f9cb8-a1cb-4801-bbe3-459980ce8476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:34.644081 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f9cb8-a1cb-4801-bbe3-459980ce8476 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:34.644373 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f9cb8-a1cb-4801-bbe3-459980ce8476 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:34.644653 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f9cb8-a1cb-4801-bbe3-459980ce8476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:34.646389 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-050f9cb8-a1cb-4801-bbe3-459980ce8476 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:34.656715 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050f9cb8-a1cb-4801-bbe3-459980ce8476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:34.658136 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050f9cb8-a1cb-4801-bbe3-459980ce8476 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:34.666253 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1369/2738] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:17:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:17:35.729997 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9092b-5f64-4113-9145-f3aa4d66ede6 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:35.738501 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9092b-5f64-4113-9145-f3aa4d66ede6 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:35.742070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f884c5-eee6-4426-bc51-1d7dc793ced0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:35.742694 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f884c5-eee6-4426-bc51-1d7dc793ced0 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:35.742977 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f884c5-eee6-4426-bc51-1d7dc793ced0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:35.743251 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f884c5-eee6-4426-bc51-1d7dc793ced0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:35.745001 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-08f884c5-eee6-4426-bc51-1d7dc793ced0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:35.806230 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f884c5-eee6-4426-bc51-1d7dc793ced0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:35.807655 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f884c5-eee6-4426-bc51-1d7dc793ced0 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:35.817228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1370/2739] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:17:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:17:36.284878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5c8a2-44f3-4646-9624-f9965f5fa9c9 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:36.285421 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5c8a2-44f3-4646-9624-f9965f5fa9c9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:36.285421 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5c8a2-44f3-4646-9624-f9965f5fa9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:36.303071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-cac5c8a2-44f3-4646-9624-f9965f5fa9c9 None None] MFA Rules not processed for user `1d37f369749143ce813c693adc5f2ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:17:36.304325 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-cac5c8a2-44f3-4646-9624-f9965f5fa9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:36.361284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1370/2740] 10.222.0.44 () {58 vars in 967 bytes} [Thu Jun 23 20:17:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:17:36.402158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f884c5-eee6-4426-bc51-1d7dc793ced0 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:36.403731 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-08f884c5-eee6-4426-bc51-1d7dc793ced0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:36.477684 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f884c5-eee6-4426-bc51-1d7dc793ced0 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:36.483911 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09adbaab-48ee-43b3-8a0c-2d99d5e58feb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96db03079816461bb322633f64b9fb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:36.484633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adbaab-48ee-43b3-8a0c-2d99d5e58feb service swift] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:36.484919 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adbaab-48ee-43b3-8a0c-2d99d5e58feb service swift] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:36.485196 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09adbaab-48ee-43b3-8a0c-2d99d5e58feb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:36.494136 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09adbaab-48ee-43b3-8a0c-2d99d5e58feb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:36.495723 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09adbaab-48ee-43b3-8a0c-2d99d5e58feb service swift] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:36.505026 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1371/2741] 10.222.0.44 () {60 vars in 1384 bytes} [Thu Jun 23 20:17:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 20:17:39.673060 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5c8a2-44f3-4646-9624-f9965f5fa9c9 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:39.675421 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-cac5c8a2-44f3-4646-9624-f9965f5fa9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:39.703758 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5c8a2-44f3-4646-9624-f9965f5fa9c9 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:39.703758 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38424c6e-c979-4b6c-9675-3520209381d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:39.703758 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38424c6e-c979-4b6c-9675-3520209381d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:39.703758 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38424c6e-c979-4b6c-9675-3520209381d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:39.704111 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38424c6e-c979-4b6c-9675-3520209381d9 admin admin] PATH_INFO: `/v3/users/2c5e6ea89435497c8b1b06f42e08c82e` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:39.704513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38424c6e-c979-4b6c-9675-3520209381d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c5e6ea89435497c8b1b06f42e08c82e)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:39.706463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38424c6e-c979-4b6c-9675-3520209381d9 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:39.879852 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-38424c6e-c979-4b6c-9675-3520209381d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c5e6ea89435497c8b1b06f42e08c82e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:39.880483 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-38424c6e-c979-4b6c-9675-3520209381d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c5e6ea89435497c8b1b06f42e08c82e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:39.966584 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-38424c6e-c979-4b6c-9675-3520209381d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c5e6ea89435497c8b1b06f42e08c82e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:39.967229 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-38424c6e-c979-4b6c-9675-3520209381d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c5e6ea89435497c8b1b06f42e08c82e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:39.980571 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-38424c6e-c979-4b6c-9675-3520209381d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c5e6ea89435497c8b1b06f42e08c82e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:39.994056 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1371/2742] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:17:39 2022] DELETE /identity/v3/users/2c5e6ea89435497c8b1b06f42e08c82e => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:17:40.238209 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09adbaab-48ee-43b3-8a0c-2d99d5e58feb service swift] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:40.253112 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09adbaab-48ee-43b3-8a0c-2d99d5e58feb service swift] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:40.346529 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd756b2-5c0c-4f95-9b2e-35b3344b608a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:40.348216 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd756b2-5c0c-4f95-9b2e-35b3344b608a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:40.348820 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd756b2-5c0c-4f95-9b2e-35b3344b608a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:40.349410 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd756b2-5c0c-4f95-9b2e-35b3344b608a admin admin] PATH_INFO: `/v3/projects/ed44d3eff7af4bd1bc339244e2bc2fae` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:40.351400 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd756b2-5c0c-4f95-9b2e-35b3344b608a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed44d3eff7af4bd1bc339244e2bc2fae)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:40.354082 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd756b2-5c0c-4f95-9b2e-35b3344b608a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:40.504809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:17:40.523297 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cbd756b2-5c0c-4f95-9b2e-35b3344b608a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed44d3eff7af4bd1bc339244e2bc2fae', 'request_id': 'req-cbd756b2-5c0c-4f95-9b2e-35b3344b608a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:40.530846 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cbd756b2-5c0c-4f95-9b2e-35b3344b608a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed44d3eff7af4bd1bc339244e2bc2fae', 'request_id': 'req-cbd756b2-5c0c-4f95-9b2e-35b3344b608a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:40.536100 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cbd756b2-5c0c-4f95-9b2e-35b3344b608a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed44d3eff7af4bd1bc339244e2bc2fae', 'request_id': 'req-cbd756b2-5c0c-4f95-9b2e-35b3344b608a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:40.611899 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cbd756b2-5c0c-4f95-9b2e-35b3344b608a admin admin] The token cache is being invalidate because project ed44d3eff7af4bd1bc339244e2bc2fae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:17:40.612191 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cbd756b2-5c0c-4f95-9b2e-35b3344b608a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:40.613000 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cbd756b2-5c0c-4f95-9b2e-35b3344b608a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:40.617665 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1372/2743] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:17:40 2022] DELETE /identity/v3/projects/ed44d3eff7af4bd1bc339244e2bc2fae => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:17:40.620084 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd32cce-cc24-482e-a744-c45164eaa68f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:40.620406 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd32cce-cc24-482e-a744-c45164eaa68f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:40.620690 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd32cce-cc24-482e-a744-c45164eaa68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:40.658249 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-edd32cce-cc24-482e-a744-c45164eaa68f None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:17:40.659606 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-edd32cce-cc24-482e-a744-c45164eaa68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:40.743502 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1372/2744] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:17:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:17:40.752086 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1097248-271c-4579-9480-06158f881ee0 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:40.752615 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1097248-271c-4579-9480-06158f881ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:40.753100 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1097248-271c-4579-9480-06158f881ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:40.773935 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e1097248-271c-4579-9480-06158f881ee0 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:17:40.775438 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e1097248-271c-4579-9480-06158f881ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:40.788535 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1373/2745] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:17:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:17:40.790863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd32cce-cc24-482e-a744-c45164eaa68f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:40.797554 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:17:40.801598 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd32cce-cc24-482e-a744-c45164eaa68f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:40.806339 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726c98f-f2ad-4edf-8c7c-7be61e73c639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:40.806993 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726c98f-f2ad-4edf-8c7c-7be61e73c639 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:40.807274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726c98f-f2ad-4edf-8c7c-7be61e73c639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:40.807663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3726c98f-f2ad-4edf-8c7c-7be61e73c639 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:40.808491 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3726c98f-f2ad-4edf-8c7c-7be61e73c639 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:17:40.808864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3726c98f-f2ad-4edf-8c7c-7be61e73c639 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:40.810572 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3726c98f-f2ad-4edf-8c7c-7be61e73c639 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:40.822280 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1373/2746] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:17:40.826659 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1097248-271c-4579-9480-06158f881ee0 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:40.836830 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1097248-271c-4579-9480-06158f881ee0 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:40.842203 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ff2da-417e-48b0-85b8-ed05239e911d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:40.845897 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ff2da-417e-48b0-85b8-ed05239e911d admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:40.845897 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ff2da-417e-48b0-85b8-ed05239e911d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:40.845897 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ff2da-417e-48b0-85b8-ed05239e911d admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:40.845897 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ff2da-417e-48b0-85b8-ed05239e911d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:40.846260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ff2da-417e-48b0-85b8-ed05239e911d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:40.928289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1374/2747] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:17:40 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:17:40.932500 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726c98f-f2ad-4edf-8c7c-7be61e73c639 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:40.942665 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3726c98f-f2ad-4edf-8c7c-7be61e73c639 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:41.009546 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5435c4-0cd8-4d3d-b281-c06b1b45c94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:41.010219 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5435c4-0cd8-4d3d-b281-c06b1b45c94f admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:41.010505 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5435c4-0cd8-4d3d-b281-c06b1b45c94f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:41.010775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5435c4-0cd8-4d3d-b281-c06b1b45c94f admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:41.011478 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5435c4-0cd8-4d3d-b281-c06b1b45c94f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:41.014272 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5435c4-0cd8-4d3d-b281-c06b1b45c94f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:41.096183 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1374/2748] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:17:40 2022] POST /identity/v3/users => generated 292 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:17:41.100671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ff2da-417e-48b0-85b8-ed05239e911d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:41.124135 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ff2da-417e-48b0-85b8-ed05239e911d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:41.142912 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f49065-0a88-48e8-b967-64ee2d687959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:41.143593 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f49065-0a88-48e8-b967-64ee2d687959 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:41.144025 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f49065-0a88-48e8-b967-64ee2d687959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:41.144357 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f49065-0a88-48e8-b967-64ee2d687959 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:41.145124 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f49065-0a88-48e8-b967-64ee2d687959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:41.146750 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f49065-0a88-48e8-b967-64ee2d687959 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:41.158528 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1375/2749] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:17:41.164013 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5435c4-0cd8-4d3d-b281-c06b1b45c94f admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:41.178186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5435c4-0cd8-4d3d-b281-c06b1b45c94f admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:41.183377 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45496723-1685-4853-aae5-71458af547e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:41.184834 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45496723-1685-4853-aae5-71458af547e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:41.185348 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45496723-1685-4853-aae5-71458af547e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:41.185702 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45496723-1685-4853-aae5-71458af547e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:41.187102 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45496723-1685-4853-aae5-71458af547e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:41.191403 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45496723-1685-4853-aae5-71458af547e1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:41.202253 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1375/2750] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:17:41.206418 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f49065-0a88-48e8-b967-64ee2d687959 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:41.220058 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f49065-0a88-48e8-b967-64ee2d687959 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:41.229611 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55460a66-0c27-4a34-accd-217564dcc3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:41.230970 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55460a66-0c27-4a34-accd-217564dcc3ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:41.231972 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55460a66-0c27-4a34-accd-217564dcc3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:41.232456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55460a66-0c27-4a34-accd-217564dcc3ea admin admin] PATH_INFO: `/v3/projects/27e2261c5de543bfbcadba9b458cd6c0/users/6a3bbb5c6332445e86349e4952e0742c/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:41.257468 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55460a66-0c27-4a34-accd-217564dcc3ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=27e2261c5de543bfbcadba9b458cd6c0, user_id=6a3bbb5c6332445e86349e4952e0742c, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:41.262193 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55460a66-0c27-4a34-accd-217564dcc3ea admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:41.332225 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1376/2751] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:17:41 2022] PUT /identity/v3/projects/27e2261c5de543bfbcadba9b458cd6c0/users/6a3bbb5c6332445e86349e4952e0742c/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:17:41.343245 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911800e-e258-42bc-9f62-b4d90c2d8d86 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:41.343591 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911800e-e258-42bc-9f62-b4d90c2d8d86 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:41.343877 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911800e-e258-42bc-9f62-b4d90c2d8d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:41.372916 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3911800e-e258-42bc-9f62-b4d90c2d8d86 None None] MFA Rules not processed for user `6a3bbb5c6332445e86349e4952e0742c`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:17:41.374306 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3911800e-e258-42bc-9f62-b4d90c2d8d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:41.458651 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1376/2752] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:17:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:17:41.471010 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55460a66-0c27-4a34-accd-217564dcc3ea admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:41.472965 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-55460a66-0c27-4a34-accd-217564dcc3ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:41.555298 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55460a66-0c27-4a34-accd-217564dcc3ea admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:41.561801 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db872bb-af28-41c2-95e4-969c7e4ad6dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:41.562565 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db872bb-af28-41c2-95e4-969c7e4ad6dc service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:41.562845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db872bb-af28-41c2-95e4-969c7e4ad6dc service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:41.563130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db872bb-af28-41c2-95e4-969c7e4ad6dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:41.571749 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db872bb-af28-41c2-95e4-969c7e4ad6dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:41.573413 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db872bb-af28-41c2-95e4-969c7e4ad6dc service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:41.586121 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1377/2753] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:17:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:17:45.278983 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3911800e-e258-42bc-9f62-b4d90c2d8d86 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:45.280492 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3911800e-e258-42bc-9f62-b4d90c2d8d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:45.345076 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3911800e-e258-42bc-9f62-b4d90c2d8d86 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:45.351274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b131d7a-441e-4951-b783-46e56c02f1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:45.352140 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b131d7a-441e-4951-b783-46e56c02f1de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:45.352449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b131d7a-441e-4951-b783-46e56c02f1de admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:45.352729 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b131d7a-441e-4951-b783-46e56c02f1de admin admin] PATH_INFO: `/v3/users/a9fe5380b3974516ac4e5b2fd9284348` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:45.353908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b131d7a-441e-4951-b783-46e56c02f1de admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9fe5380b3974516ac4e5b2fd9284348)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:45.358284 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b131d7a-441e-4951-b783-46e56c02f1de admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:45.359880 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:17:45.527613 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2b131d7a-441e-4951-b783-46e56c02f1de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9fe5380b3974516ac4e5b2fd9284348'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:45.528192 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2b131d7a-441e-4951-b783-46e56c02f1de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9fe5380b3974516ac4e5b2fd9284348'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:45.599667 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2b131d7a-441e-4951-b783-46e56c02f1de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9fe5380b3974516ac4e5b2fd9284348'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:45.600275 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2b131d7a-441e-4951-b783-46e56c02f1de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9fe5380b3974516ac4e5b2fd9284348'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:45.613401 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2b131d7a-441e-4951-b783-46e56c02f1de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9fe5380b3974516ac4e5b2fd9284348'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:45.628187 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1377/2754] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:17:45 2022] DELETE /identity/v3/users/a9fe5380b3974516ac4e5b2fd9284348 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:17:45.875974 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db872bb-af28-41c2-95e4-969c7e4ad6dc service nova] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:45.886783 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db872bb-af28-41c2-95e4-969c7e4ad6dc service nova] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:45.921262 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:17:45.945352 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c6535-9b29-4eba-9895-4a0f409ffe8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:45.946196 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c6535-9b29-4eba-9895-4a0f409ffe8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:45.946472 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c6535-9b29-4eba-9895-4a0f409ffe8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:45.946742 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c6535-9b29-4eba-9895-4a0f409ffe8b admin admin] PATH_INFO: `/v3/projects/a2028a4056254026b31f412942d1f576` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:45.947893 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c6535-9b29-4eba-9895-4a0f409ffe8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2028a4056254026b31f412942d1f576)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:45.950411 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c6535-9b29-4eba-9895-4a0f409ffe8b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:46.064995 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-012c6535-9b29-4eba-9895-4a0f409ffe8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2028a4056254026b31f412942d1f576', 'request_id': 'req-012c6535-9b29-4eba-9895-4a0f409ffe8b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:46.071619 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-012c6535-9b29-4eba-9895-4a0f409ffe8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2028a4056254026b31f412942d1f576', 'request_id': 'req-012c6535-9b29-4eba-9895-4a0f409ffe8b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:46.077037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-012c6535-9b29-4eba-9895-4a0f409ffe8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2028a4056254026b31f412942d1f576', 'request_id': 'req-012c6535-9b29-4eba-9895-4a0f409ffe8b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:46.136637 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-012c6535-9b29-4eba-9895-4a0f409ffe8b admin admin] The token cache is being invalidate because project a2028a4056254026b31f412942d1f576 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:17:46.136887 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-012c6535-9b29-4eba-9895-4a0f409ffe8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:46.138037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-012c6535-9b29-4eba-9895-4a0f409ffe8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:46.139687 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1378/2755] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:17:45 2022] DELETE /identity/v3/projects/a2028a4056254026b31f412942d1f576 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:17:46.143641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b131d7a-441e-4951-b783-46e56c02f1de admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:46.145265 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2b131d7a-441e-4951-b783-46e56c02f1de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:46.158832 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b131d7a-441e-4951-b783-46e56c02f1de admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:46.213752 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c476ede-5dc5-4045-90ae-ae9a4d0cca29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:46.214614 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c476ede-5dc5-4045-90ae-ae9a4d0cca29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:46.214895 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c476ede-5dc5-4045-90ae-ae9a4d0cca29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:46.215168 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c476ede-5dc5-4045-90ae-ae9a4d0cca29 admin admin] PATH_INFO: `/v3/users/c24fddf0b70441c1947610ce9578c98c` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:46.217005 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c476ede-5dc5-4045-90ae-ae9a4d0cca29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c24fddf0b70441c1947610ce9578c98c)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:46.218924 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c476ede-5dc5-4045-90ae-ae9a4d0cca29 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:46.385909 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1c476ede-5dc5-4045-90ae-ae9a4d0cca29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c24fddf0b70441c1947610ce9578c98c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:46.386549 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1c476ede-5dc5-4045-90ae-ae9a4d0cca29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c24fddf0b70441c1947610ce9578c98c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:46.435121 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1c476ede-5dc5-4045-90ae-ae9a4d0cca29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c24fddf0b70441c1947610ce9578c98c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:46.435676 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1c476ede-5dc5-4045-90ae-ae9a4d0cca29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c24fddf0b70441c1947610ce9578c98c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:46.447903 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1c476ede-5dc5-4045-90ae-ae9a4d0cca29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c24fddf0b70441c1947610ce9578c98c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:46.455818 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:17:46.461041 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1378/2756] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:17:46 2022] DELETE /identity/v3/users/c24fddf0b70441c1947610ce9578c98c => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:17:46.722324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c6535-9b29-4eba-9895-4a0f409ffe8b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:46.723741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-012c6535-9b29-4eba-9895-4a0f409ffe8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:46.738060 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c6535-9b29-4eba-9895-4a0f409ffe8b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:46.802733 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb454ad8-b11a-4b5e-8a53-db04db7e1ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:46.803598 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb454ad8-b11a-4b5e-8a53-db04db7e1ab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:46.803879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb454ad8-b11a-4b5e-8a53-db04db7e1ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:46.804149 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb454ad8-b11a-4b5e-8a53-db04db7e1ab9 admin admin] PATH_INFO: `/v3/projects/29ed429db99d48b285a2e295ec0073bd` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:46.805430 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb454ad8-b11a-4b5e-8a53-db04db7e1ab9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29ed429db99d48b285a2e295ec0073bd)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:46.807306 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb454ad8-b11a-4b5e-8a53-db04db7e1ab9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:46.927442 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cb454ad8-b11a-4b5e-8a53-db04db7e1ab9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29ed429db99d48b285a2e295ec0073bd', 'request_id': 'req-cb454ad8-b11a-4b5e-8a53-db04db7e1ab9'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:46.933738 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cb454ad8-b11a-4b5e-8a53-db04db7e1ab9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29ed429db99d48b285a2e295ec0073bd', 'request_id': 'req-cb454ad8-b11a-4b5e-8a53-db04db7e1ab9'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:46.939859 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cb454ad8-b11a-4b5e-8a53-db04db7e1ab9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29ed429db99d48b285a2e295ec0073bd', 'request_id': 'req-cb454ad8-b11a-4b5e-8a53-db04db7e1ab9'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:46.944478 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:17:47.000522 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cb454ad8-b11a-4b5e-8a53-db04db7e1ab9 admin admin] The token cache is being invalidate because project 29ed429db99d48b285a2e295ec0073bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:17:47.000730 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cb454ad8-b11a-4b5e-8a53-db04db7e1ab9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:47.001328 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cb454ad8-b11a-4b5e-8a53-db04db7e1ab9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:47.003107 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1379/2757] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:17:46 2022] DELETE /identity/v3/projects/29ed429db99d48b285a2e295ec0073bd => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:17:47.007777 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fc3cd-4c12-4069-8561-16bee1632b9e None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:47.008065 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fc3cd-4c12-4069-8561-16bee1632b9e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:47.008383 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fc3cd-4c12-4069-8561-16bee1632b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:47.026910 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-002fc3cd-4c12-4069-8561-16bee1632b9e None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:17:47.028163 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-002fc3cd-4c12-4069-8561-16bee1632b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:47.105673 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1379/2758] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:17:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:17:47.113376 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac8e4e-2f61-446e-9cd3-f309750a2e84 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:47.113664 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac8e4e-2f61-446e-9cd3-f309750a2e84 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:47.113941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac8e4e-2f61-446e-9cd3-f309750a2e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:47.133051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ceac8e4e-2f61-446e-9cd3-f309750a2e84 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:17:47.134265 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ceac8e4e-2f61-446e-9cd3-f309750a2e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:47.143190 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1380/2759] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:17:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:17:47.147526 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fc3cd-4c12-4069-8561-16bee1632b9e None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:47.158939 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fc3cd-4c12-4069-8561-16bee1632b9e None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:47.163511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4c9b6-0b08-41a3-a67c-1e753da9bc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:47.164168 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4c9b6-0b08-41a3-a67c-1e753da9bc0c admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:47.164479 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4c9b6-0b08-41a3-a67c-1e753da9bc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:47.164756 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4c9b6-0b08-41a3-a67c-1e753da9bc0c admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:47.165484 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d4c9b6-0b08-41a3-a67c-1e753da9bc0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:17:47.165789 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d4c9b6-0b08-41a3-a67c-1e753da9bc0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:47.167398 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d4c9b6-0b08-41a3-a67c-1e753da9bc0c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:47.179709 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1380/2760] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:17:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:17:47.192273 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac8e4e-2f61-446e-9cd3-f309750a2e84 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:47.202432 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac8e4e-2f61-446e-9cd3-f309750a2e84 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:47.207473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31801e20-a838-4531-af6d-9fb2fb6ca135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:47.208121 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31801e20-a838-4531-af6d-9fb2fb6ca135 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:47.208422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31801e20-a838-4531-af6d-9fb2fb6ca135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:47.217473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31801e20-a838-4531-af6d-9fb2fb6ca135 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:47.217473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31801e20-a838-4531-af6d-9fb2fb6ca135 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:47.217473 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31801e20-a838-4531-af6d-9fb2fb6ca135 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:47.327510 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1381/2761] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:17:47 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:17:47.332034 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4c9b6-0b08-41a3-a67c-1e753da9bc0c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:47.343156 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4c9b6-0b08-41a3-a67c-1e753da9bc0c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:47.401750 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a02416-c411-4907-a678-7a1395a88992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:47.402552 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a02416-c411-4907-a678-7a1395a88992 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:47.402832 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a02416-c411-4907-a678-7a1395a88992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:47.403103 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a02416-c411-4907-a678-7a1395a88992 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:47.412857 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a02416-c411-4907-a678-7a1395a88992 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:47.412857 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a02416-c411-4907-a678-7a1395a88992 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:47.492853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1381/2762] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:17:47 2022] POST /identity/v3/users => generated 287 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:17:47.496427 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31801e20-a838-4531-af6d-9fb2fb6ca135 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:47.509213 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31801e20-a838-4531-af6d-9fb2fb6ca135 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:47.515454 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730fef5-3f93-4d98-917d-b134a660777b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:47.516124 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730fef5-3f93-4d98-917d-b134a660777b admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:47.516444 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730fef5-3f93-4d98-917d-b134a660777b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:47.516722 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730fef5-3f93-4d98-917d-b134a660777b admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:47.518002 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730fef5-3f93-4d98-917d-b134a660777b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:47.519675 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730fef5-3f93-4d98-917d-b134a660777b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:47.532806 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1382/2763] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:17:47.534013 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a02416-c411-4907-a678-7a1395a88992 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:47.543928 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a02416-c411-4907-a678-7a1395a88992 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:47.548541 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e9fced-8e77-4aaf-9c32-d1e58543f9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:47.549193 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9fced-8e77-4aaf-9c32-d1e58543f9b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:47.549472 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9fced-8e77-4aaf-9c32-d1e58543f9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:47.549745 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9fced-8e77-4aaf-9c32-d1e58543f9b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:47.550466 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e9fced-8e77-4aaf-9c32-d1e58543f9b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:47.552058 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e9fced-8e77-4aaf-9c32-d1e58543f9b1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:47.562431 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1382/2764] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:17:47.566571 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730fef5-3f93-4d98-917d-b134a660777b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:47.577032 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730fef5-3f93-4d98-917d-b134a660777b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:47.581586 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583e84d-ba39-45c3-ac30-360854540d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:47.582449 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583e84d-ba39-45c3-ac30-360854540d90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:47.582738 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583e84d-ba39-45c3-ac30-360854540d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:47.583013 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583e84d-ba39-45c3-ac30-360854540d90 admin admin] PATH_INFO: `/v3/projects/47ee9522c6214257b59f3a350220dc0b/users/3fa08242da514c0d9fa719830dcfc760/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:47.601261 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583e84d-ba39-45c3-ac30-360854540d90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47ee9522c6214257b59f3a350220dc0b, user_id=3fa08242da514c0d9fa719830dcfc760, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:47.604924 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583e84d-ba39-45c3-ac30-360854540d90 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:47.703438 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1383/2765] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:17:47 2022] PUT /identity/v3/projects/47ee9522c6214257b59f3a350220dc0b/users/3fa08242da514c0d9fa719830dcfc760/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:17:47.708581 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36480977-107e-48b0-87a6-92ae8fbf6e0a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:47.708873 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36480977-107e-48b0-87a6-92ae8fbf6e0a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:47.709174 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36480977-107e-48b0-87a6-92ae8fbf6e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:47.727370 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-36480977-107e-48b0-87a6-92ae8fbf6e0a None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:17:47.728619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-36480977-107e-48b0-87a6-92ae8fbf6e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:47.787851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1383/2766] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:17:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:17:47.797881 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583e84d-ba39-45c3-ac30-360854540d90 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:47.799741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c583e84d-ba39-45c3-ac30-360854540d90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:47.855712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583e84d-ba39-45c3-ac30-360854540d90 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:47.859883 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d12fe7-600d-4f66-b5d1-e88ad5a26538 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:47.860639 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d12fe7-600d-4f66-b5d1-e88ad5a26538 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:47.860925 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d12fe7-600d-4f66-b5d1-e88ad5a26538 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:47.861203 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d12fe7-600d-4f66-b5d1-e88ad5a26538 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:47.870391 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d12fe7-600d-4f66-b5d1-e88ad5a26538 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:47.871825 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d12fe7-600d-4f66-b5d1-e88ad5a26538 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:47.882551 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1384/2767] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:17:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:17:51.847435 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36480977-107e-48b0-87a6-92ae8fbf6e0a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:51.848656 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-36480977-107e-48b0-87a6-92ae8fbf6e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:51.857744 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:17:51.861269 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36480977-107e-48b0-87a6-92ae8fbf6e0a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:51.866337 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c0e59-d44f-42cb-8ead-4f1b28eefd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:51.867240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c0e59-d44f-42cb-8ead-4f1b28eefd28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:51.867511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c0e59-d44f-42cb-8ead-4f1b28eefd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:51.867785 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c0e59-d44f-42cb-8ead-4f1b28eefd28 admin admin] PATH_INFO: `/v3/users/6a3bbb5c6332445e86349e4952e0742c` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:51.869005 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c0e59-d44f-42cb-8ead-4f1b28eefd28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a3bbb5c6332445e86349e4952e0742c)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:51.871757 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c0e59-d44f-42cb-8ead-4f1b28eefd28 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:52.080277 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b66c0e59-d44f-42cb-8ead-4f1b28eefd28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a3bbb5c6332445e86349e4952e0742c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:52.080975 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b66c0e59-d44f-42cb-8ead-4f1b28eefd28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a3bbb5c6332445e86349e4952e0742c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:52.188904 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b66c0e59-d44f-42cb-8ead-4f1b28eefd28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a3bbb5c6332445e86349e4952e0742c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:52.189641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b66c0e59-d44f-42cb-8ead-4f1b28eefd28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a3bbb5c6332445e86349e4952e0742c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:52.213266 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b66c0e59-d44f-42cb-8ead-4f1b28eefd28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a3bbb5c6332445e86349e4952e0742c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:52.227084 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1384/2768] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:17:51 2022] DELETE /identity/v3/users/6a3bbb5c6332445e86349e4952e0742c => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:17:52.231733 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c5a90-d28f-4c88-9340-e1b3bbc00128 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:52.233189 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c5a90-d28f-4c88-9340-e1b3bbc00128 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:52.233189 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c5a90-d28f-4c88-9340-e1b3bbc00128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:52.252671 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f72c5a90-d28f-4c88-9340-e1b3bbc00128 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:17:52.264807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f72c5a90-d28f-4c88-9340-e1b3bbc00128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:52.411863 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1385/2769] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:17:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:17:52.423965 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c0e59-d44f-42cb-8ead-4f1b28eefd28 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:52.426419 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b66c0e59-d44f-42cb-8ead-4f1b28eefd28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:52.509897 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c0e59-d44f-42cb-8ead-4f1b28eefd28 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:52.511828 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe99d31-4a82-4177-bb29-3872c127bebf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:52.512527 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe99d31-4a82-4177-bb29-3872c127bebf service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:52.512807 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe99d31-4a82-4177-bb29-3872c127bebf service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:52.513119 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe99d31-4a82-4177-bb29-3872c127bebf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:52.525267 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe99d31-4a82-4177-bb29-3872c127bebf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:52.526812 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe99d31-4a82-4177-bb29-3872c127bebf service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:52.540768 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1385/2770] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:17:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:17:53.058786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbe431-e170-4ffa-8bc4-12cad41cc74d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:53.059103 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbe431-e170-4ffa-8bc4-12cad41cc74d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:53.059653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbe431-e170-4ffa-8bc4-12cad41cc74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:53.091634 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-46fbe431-e170-4ffa-8bc4-12cad41cc74d None None] MFA Rules not processed for user `3fa08242da514c0d9fa719830dcfc760`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:17:53.092916 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-46fbe431-e170-4ffa-8bc4-12cad41cc74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:53.121579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:17:53.166876 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1386/2771] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:17:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:17:53.168096 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe99d31-4a82-4177-bb29-3872c127bebf service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:53.169786 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0fe99d31-4a82-4177-bb29-3872c127bebf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:53.169918 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbe431-e170-4ffa-8bc4-12cad41cc74d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:53.171441 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-46fbe431-e170-4ffa-8bc4-12cad41cc74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:53.187941 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fbe431-e170-4ffa-8bc4-12cad41cc74d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:53.191060 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf4657-dd69-4036-be1c-f7e8e44a4be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:53.192033 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf4657-dd69-4036-be1c-f7e8e44a4be3 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:53.192348 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf4657-dd69-4036-be1c-f7e8e44a4be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:53.192705 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf4657-dd69-4036-be1c-f7e8e44a4be3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:53.193381 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf4657-dd69-4036-be1c-f7e8e44a4be3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:53.196006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cf4657-dd69-4036-be1c-f7e8e44a4be3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:53.196444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe99d31-4a82-4177-bb29-3872c127bebf service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:53.202713 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7058b1-fd5b-4f44-b68b-eaa8c3b271a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:53.203573 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7058b1-fd5b-4f44-b68b-eaa8c3b271a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:53.203875 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7058b1-fd5b-4f44-b68b-eaa8c3b271a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:53.204256 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7058b1-fd5b-4f44-b68b-eaa8c3b271a5 admin admin] PATH_INFO: `/v3/projects/27e2261c5de543bfbcadba9b458cd6c0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:53.205410 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7058b1-fd5b-4f44-b68b-eaa8c3b271a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27e2261c5de543bfbcadba9b458cd6c0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:53.207344 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7058b1-fd5b-4f44-b68b-eaa8c3b271a5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:53.293905 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1387/2772] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:17:53 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:17:53.298008 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf4657-dd69-4036-be1c-f7e8e44a4be3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:53.325226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4d7058b1-fd5b-4f44-b68b-eaa8c3b271a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27e2261c5de543bfbcadba9b458cd6c0', 'request_id': 'req-4d7058b1-fd5b-4f44-b68b-eaa8c3b271a5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:53.367709 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cf4657-dd69-4036-be1c-f7e8e44a4be3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:53.371900 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4d7058b1-fd5b-4f44-b68b-eaa8c3b271a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27e2261c5de543bfbcadba9b458cd6c0', 'request_id': 'req-4d7058b1-fd5b-4f44-b68b-eaa8c3b271a5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:53.383315 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4d7058b1-fd5b-4f44-b68b-eaa8c3b271a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27e2261c5de543bfbcadba9b458cd6c0', 'request_id': 'req-4d7058b1-fd5b-4f44-b68b-eaa8c3b271a5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:53.397892 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4d7058b1-fd5b-4f44-b68b-eaa8c3b271a5 admin admin] The token cache is being invalidate because project 27e2261c5de543bfbcadba9b458cd6c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:17:53.398380 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4d7058b1-fd5b-4f44-b68b-eaa8c3b271a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:53.398887 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4d7058b1-fd5b-4f44-b68b-eaa8c3b271a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:53.404111 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1386/2773] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:17:53 2022] DELETE /identity/v3/projects/27e2261c5de543bfbcadba9b458cd6c0 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:17:53.405610 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d9d81-4a2b-42b9-a773-8e84858e3660 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:53.405904 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d9d81-4a2b-42b9-a773-8e84858e3660 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:53.406187 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d9d81-4a2b-42b9-a773-8e84858e3660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:53.413688 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:17:53.425403 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-bb2d9d81-4a2b-42b9-a773-8e84858e3660 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:17:53.426627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bb2d9d81-4a2b-42b9-a773-8e84858e3660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:53.432433 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d888c2-4c86-4060-893a-7ebe90a62a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:53.433100 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d888c2-4c86-4060-893a-7ebe90a62a73 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:53.433382 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d888c2-4c86-4060-893a-7ebe90a62a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:53.433681 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d888c2-4c86-4060-893a-7ebe90a62a73 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:53.434369 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d888c2-4c86-4060-893a-7ebe90a62a73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:53.437020 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d888c2-4c86-4060-893a-7ebe90a62a73 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:53.488168 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1387/2774] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:17:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:17:53.495528 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4eba2-ea7f-4c87-9d24-4ccf31913397 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:53.495818 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4eba2-ea7f-4c87-9d24-4ccf31913397 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:53.496093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4eba2-ea7f-4c87-9d24-4ccf31913397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:53.515840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a6a4eba2-ea7f-4c87-9d24-4ccf31913397 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:17:53.517094 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a6a4eba2-ea7f-4c87-9d24-4ccf31913397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:53.525142 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1388/2775] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:17:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:17:53.525468 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1388/2776] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:17:53 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:17:53.529445 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4eba2-ea7f-4c87-9d24-4ccf31913397 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:53.530803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a6a4eba2-ea7f-4c87-9d24-4ccf31913397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:53.532548 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d888c2-4c86-4060-893a-7ebe90a62a73 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:53.542938 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d888c2-4c86-4060-893a-7ebe90a62a73 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:53.543237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4eba2-ea7f-4c87-9d24-4ccf31913397 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:53.548613 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf784ee1-51d9-4f19-b4b7-e13bd26cf17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:53.548853 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b064ba-dd00-4d2a-9df6-9239167e6b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:53.549257 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf784ee1-51d9-4f19-b4b7-e13bd26cf17b admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:53.549378 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b064ba-dd00-4d2a-9df6-9239167e6b90 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:53.549560 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf784ee1-51d9-4f19-b4b7-e13bd26cf17b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:53.549680 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b064ba-dd00-4d2a-9df6-9239167e6b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:53.549838 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf784ee1-51d9-4f19-b4b7-e13bd26cf17b admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:53.549954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b064ba-dd00-4d2a-9df6-9239167e6b90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:53.550565 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf784ee1-51d9-4f19-b4b7-e13bd26cf17b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:17:53.550668 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b064ba-dd00-4d2a-9df6-9239167e6b90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:53.550873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf784ee1-51d9-4f19-b4b7-e13bd26cf17b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:53.552207 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b064ba-dd00-4d2a-9df6-9239167e6b90 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:53.552540 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf784ee1-51d9-4f19-b4b7-e13bd26cf17b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:53.565714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1389/2777] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:17:53.571745 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1389/2778] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:17:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:17:53.571745 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf784ee1-51d9-4f19-b4b7-e13bd26cf17b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:53.571954 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b064ba-dd00-4d2a-9df6-9239167e6b90 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:53.579127 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b064ba-dd00-4d2a-9df6-9239167e6b90 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:53.579672 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf784ee1-51d9-4f19-b4b7-e13bd26cf17b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:53.584284 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791fc83-74b4-4b5a-8b8c-ab17d1d82a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:53.584676 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198fc2d0-333e-41c9-b9fe-c105157f43d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:53.584931 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791fc83-74b4-4b5a-8b8c-ab17d1d82a29 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:53.585210 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791fc83-74b4-4b5a-8b8c-ab17d1d82a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:53.585483 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791fc83-74b4-4b5a-8b8c-ab17d1d82a29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:53.585875 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198fc2d0-333e-41c9-b9fe-c105157f43d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:53.586178 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198fc2d0-333e-41c9-b9fe-c105157f43d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:53.586422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791fc83-74b4-4b5a-8b8c-ab17d1d82a29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:53.586535 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198fc2d0-333e-41c9-b9fe-c105157f43d9 admin admin] PATH_INFO: `/v3/projects/4b19b714218d4bea9818c0162a66b1b4/users/2ebdcce97d7c49aa8edaf8d2fb069962/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:53.588459 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791fc83-74b4-4b5a-8b8c-ab17d1d82a29 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:53.603818 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198fc2d0-333e-41c9-b9fe-c105157f43d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b19b714218d4bea9818c0162a66b1b4, user_id=2ebdcce97d7c49aa8edaf8d2fb069962, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:53.606608 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198fc2d0-333e-41c9-b9fe-c105157f43d9 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:53.665097 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1390/2779] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:17:53 2022] POST /identity/v3/projects => generated 371 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:17:53.669289 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1390/2780] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:17:53 2022] PUT /identity/v3/projects/4b19b714218d4bea9818c0162a66b1b4/users/2ebdcce97d7c49aa8edaf8d2fb069962/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:17:53.672341 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791fc83-74b4-4b5a-8b8c-ab17d1d82a29 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:53.673134 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198fc2d0-333e-41c9-b9fe-c105157f43d9 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:53.678128 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:17:53.738630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198fc2d0-333e-41c9-b9fe-c105157f43d9 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:53.739002 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791fc83-74b4-4b5a-8b8c-ab17d1d82a29 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:53.743160 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e3295-6e7c-4f30-8ba5-0d584f8f7c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:53.743544 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f6959-e2ad-47e8-80f0-94dba6bce12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:53.743820 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e3295-6e7c-4f30-8ba5-0d584f8f7c55 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:53.744098 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e3295-6e7c-4f30-8ba5-0d584f8f7c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:53.744223 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f6959-e2ad-47e8-80f0-94dba6bce12b admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:53.744380 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e3295-6e7c-4f30-8ba5-0d584f8f7c55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:53.744551 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f6959-e2ad-47e8-80f0-94dba6bce12b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:53.744826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f6959-e2ad-47e8-80f0-94dba6bce12b admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:53.745102 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e3295-6e7c-4f30-8ba5-0d584f8f7c55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:53.745543 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f6959-e2ad-47e8-80f0-94dba6bce12b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:53.746699 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e3295-6e7c-4f30-8ba5-0d584f8f7c55 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:53.747472 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f6959-e2ad-47e8-80f0-94dba6bce12b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:53.756669 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1391/2781] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:17:53.760663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e3295-6e7c-4f30-8ba5-0d584f8f7c55 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:53.770646 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e3295-6e7c-4f30-8ba5-0d584f8f7c55 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:53.775324 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5369cba1-f917-4e38-a7f8-4ce9db343365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:53.776209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369cba1-f917-4e38-a7f8-4ce9db343365 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:53.776523 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369cba1-f917-4e38-a7f8-4ce9db343365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:53.776802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5369cba1-f917-4e38-a7f8-4ce9db343365 admin admin] PATH_INFO: `/v3/projects/4b19b714218d4bea9818c0162a66b1b4/users/2ebdcce97d7c49aa8edaf8d2fb069962/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:53.778758 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5369cba1-f917-4e38-a7f8-4ce9db343365 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b19b714218d4bea9818c0162a66b1b4, user_id=2ebdcce97d7c49aa8edaf8d2fb069962, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:53.782064 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5369cba1-f917-4e38-a7f8-4ce9db343365 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:53.792214 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1392/2782] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:17:53 2022] PUT /identity/v3/projects/4b19b714218d4bea9818c0162a66b1b4/users/2ebdcce97d7c49aa8edaf8d2fb069962/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:17:53.824404 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1391/2783] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:17:53 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:17:53.828594 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5369cba1-f917-4e38-a7f8-4ce9db343365 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:53.886642 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:17:53.904397 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5369cba1-f917-4e38-a7f8-4ce9db343365 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:53.909238 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a95c9a-64fb-483f-b9b8-02428d33fda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:53.909922 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a95c9a-64fb-483f-b9b8-02428d33fda5 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:53.910200 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a95c9a-64fb-483f-b9b8-02428d33fda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:53.910467 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a95c9a-64fb-483f-b9b8-02428d33fda5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:53.911415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a95c9a-64fb-483f-b9b8-02428d33fda5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:53.913900 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a95c9a-64fb-483f-b9b8-02428d33fda5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:53.926265 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1393/2784] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:17:53.930330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f6959-e2ad-47e8-80f0-94dba6bce12b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:53.941721 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f6959-e2ad-47e8-80f0-94dba6bce12b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:53.947309 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ac47c-1dee-4a62-ac8b-42c3e50cf7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:53.947763 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ac47c-1dee-4a62-ac8b-42c3e50cf7d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:53.948200 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ac47c-1dee-4a62-ac8b-42c3e50cf7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:53.948546 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ac47c-1dee-4a62-ac8b-42c3e50cf7d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:53.949320 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ac47c-1dee-4a62-ac8b-42c3e50cf7d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:53.951026 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ac47c-1dee-4a62-ac8b-42c3e50cf7d1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:53.965106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1392/2785] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:17:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:17:53.971100 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a95c9a-64fb-483f-b9b8-02428d33fda5 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:53.981807 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a95c9a-64fb-483f-b9b8-02428d33fda5 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:53.987296 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef515c16-c7d1-428a-a95f-385c1da391bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:53.988194 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef515c16-c7d1-428a-a95f-385c1da391bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:53.988514 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef515c16-c7d1-428a-a95f-385c1da391bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:53.988859 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef515c16-c7d1-428a-a95f-385c1da391bb admin admin] PATH_INFO: `/v3/projects/7a72eea3466544deb83ea31091f733b1/users/02e2b6d9e34845d7b6521dd74a63c701/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:54.006229 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef515c16-c7d1-428a-a95f-385c1da391bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a72eea3466544deb83ea31091f733b1, user_id=02e2b6d9e34845d7b6521dd74a63c701, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:54.009473 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef515c16-c7d1-428a-a95f-385c1da391bb admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:54.082033 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1394/2786] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:17:53 2022] PUT /identity/v3/projects/7a72eea3466544deb83ea31091f733b1/users/02e2b6d9e34845d7b6521dd74a63c701/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:17:54.092456 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15502e-82b4-4929-91f3-09c5bf0aa1c2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:54.092756 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15502e-82b4-4929-91f3-09c5bf0aa1c2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:54.093028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15502e-82b4-4929-91f3-09c5bf0aa1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:54.123407 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5c15502e-82b4-4929-91f3-09c5bf0aa1c2 None None] MFA Rules not processed for user `02e2b6d9e34845d7b6521dd74a63c701`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:17:54.124661 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5c15502e-82b4-4929-91f3-09c5bf0aa1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:54.206658 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1393/2787] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:17:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:17:54.218693 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef515c16-c7d1-428a-a95f-385c1da391bb admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:54.220539 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ef515c16-c7d1-428a-a95f-385c1da391bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:54.345247 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef515c16-c7d1-428a-a95f-385c1da391bb admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:54.351697 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2379e8-0159-4410-a5df-e4a43478c315 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:54.356482 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2379e8-0159-4410-a5df-e4a43478c315 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:54.356482 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2379e8-0159-4410-a5df-e4a43478c315 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:54.356482 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2379e8-0159-4410-a5df-e4a43478c315 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:54.363880 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2379e8-0159-4410-a5df-e4a43478c315 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:54.366694 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2379e8-0159-4410-a5df-e4a43478c315 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:54.376607 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1395/2788] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:17:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:17:54.638859 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15502e-82b4-4929-91f3-09c5bf0aa1c2 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:54.705760 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15502e-82b4-4929-91f3-09c5bf0aa1c2 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:54.710449 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066fa14d-146d-4906-97c4-0d7c8ae35dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:54.711785 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066fa14d-146d-4906-97c4-0d7c8ae35dcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:54.715913 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066fa14d-146d-4906-97c4-0d7c8ae35dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:54.715913 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066fa14d-146d-4906-97c4-0d7c8ae35dcc admin admin] PATH_INFO: `/v3/users/02e2b6d9e34845d7b6521dd74a63c701` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:54.715913 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066fa14d-146d-4906-97c4-0d7c8ae35dcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=02e2b6d9e34845d7b6521dd74a63c701)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:54.715913 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066fa14d-146d-4906-97c4-0d7c8ae35dcc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:54.730165 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:17:54.856220 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-066fa14d-146d-4906-97c4-0d7c8ae35dcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02e2b6d9e34845d7b6521dd74a63c701'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:54.856982 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-066fa14d-146d-4906-97c4-0d7c8ae35dcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02e2b6d9e34845d7b6521dd74a63c701'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:54.869622 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-066fa14d-146d-4906-97c4-0d7c8ae35dcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02e2b6d9e34845d7b6521dd74a63c701'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:54.876969 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-066fa14d-146d-4906-97c4-0d7c8ae35dcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02e2b6d9e34845d7b6521dd74a63c701'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:54.949180 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-066fa14d-146d-4906-97c4-0d7c8ae35dcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02e2b6d9e34845d7b6521dd74a63c701'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:54.961556 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75ae86-a595-4e3b-b6b5-c288c090bc41 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:54.961848 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75ae86-a595-4e3b-b6b5-c288c090bc41 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:54.962568 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75ae86-a595-4e3b-b6b5-c288c090bc41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:54.969157 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1394/2789] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:17:54 2022] DELETE /identity/v3/users/02e2b6d9e34845d7b6521dd74a63c701 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:17:54.981020 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ac75ae86-a595-4e3b-b6b5-c288c090bc41 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:17:54.983225 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ac75ae86-a595-4e3b-b6b5-c288c090bc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:55.065380 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1396/2790] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:17:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:17:55.075158 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066fa14d-146d-4906-97c4-0d7c8ae35dcc admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:55.077285 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-066fa14d-146d-4906-97c4-0d7c8ae35dcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:55.139930 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066fa14d-146d-4906-97c4-0d7c8ae35dcc admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:55.149691 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe2566-aeb7-4914-aacc-313a3355b1cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:55.149691 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe2566-aeb7-4914-aacc-313a3355b1cd service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:55.149691 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe2566-aeb7-4914-aacc-313a3355b1cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:55.149691 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe2566-aeb7-4914-aacc-313a3355b1cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:55.160025 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe2566-aeb7-4914-aacc-313a3355b1cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:55.161579 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe2566-aeb7-4914-aacc-313a3355b1cd service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:55.178219 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1395/2791] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:17:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:17:55.641249 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75ae86-a595-4e3b-b6b5-c288c090bc41 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:55.643931 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ac75ae86-a595-4e3b-b6b5-c288c090bc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:55.668657 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75ae86-a595-4e3b-b6b5-c288c090bc41 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:55.676498 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90516cf6-2743-4d96-90a9-7761699735c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:55.677495 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90516cf6-2743-4d96-90a9-7761699735c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:55.677872 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90516cf6-2743-4d96-90a9-7761699735c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:55.678429 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90516cf6-2743-4d96-90a9-7761699735c5 admin admin] PATH_INFO: `/v3/projects/7a72eea3466544deb83ea31091f733b1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:55.679639 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90516cf6-2743-4d96-90a9-7761699735c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a72eea3466544deb83ea31091f733b1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:55.681736 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90516cf6-2743-4d96-90a9-7761699735c5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:55.762007 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:17:55.771315 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-90516cf6-2743-4d96-90a9-7761699735c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a72eea3466544deb83ea31091f733b1', 'request_id': 'req-90516cf6-2743-4d96-90a9-7761699735c5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:55.840268 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-90516cf6-2743-4d96-90a9-7761699735c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a72eea3466544deb83ea31091f733b1', 'request_id': 'req-90516cf6-2743-4d96-90a9-7761699735c5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:55.849995 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-90516cf6-2743-4d96-90a9-7761699735c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a72eea3466544deb83ea31091f733b1', 'request_id': 'req-90516cf6-2743-4d96-90a9-7761699735c5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:55.865610 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-90516cf6-2743-4d96-90a9-7761699735c5 admin admin] The token cache is being invalidate because project 7a72eea3466544deb83ea31091f733b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:17:55.866031 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-90516cf6-2743-4d96-90a9-7761699735c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:55.866940 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-90516cf6-2743-4d96-90a9-7761699735c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:17:55.868748 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1397/2792] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:17:55 2022] DELETE /identity/v3/projects/7a72eea3466544deb83ea31091f733b1 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:17:55.873422 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bda01-52c2-4290-aa97-d52b9d2b1f9c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:55.873943 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bda01-52c2-4290-aa97-d52b9d2b1f9c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:55.874164 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4bda01-52c2-4290-aa97-d52b9d2b1f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:55.893330 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ed4bda01-52c2-4290-aa97-d52b9d2b1f9c None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:17:55.894563 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ed4bda01-52c2-4290-aa97-d52b9d2b1f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:55.979606 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402836c9-c58c-4203-bab1-4deeedfca5bd None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:55.980102 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402836c9-c58c-4203-bab1-4deeedfca5bd None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:55.980566 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402836c9-c58c-4203-bab1-4deeedfca5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:55.984138 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1396/2793] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:17:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:17:56.006034 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-402836c9-c58c-4203-bab1-4deeedfca5bd None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:17:56.008000 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-402836c9-c58c-4203-bab1-4deeedfca5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:56.023641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1398/2794] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:17:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:17:56.028534 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4bda01-52c2-4290-aa97-d52b9d2b1f9c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:56.042164 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4bda01-52c2-4290-aa97-d52b9d2b1f9c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:56.048963 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088e869-4d40-410e-b52b-f7dd633ba729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:56.053626 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088e869-4d40-410e-b52b-f7dd633ba729 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:56.053626 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088e869-4d40-410e-b52b-f7dd633ba729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:56.053626 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088e869-4d40-410e-b52b-f7dd633ba729 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:56.053626 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5088e869-4d40-410e-b52b-f7dd633ba729 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:17:56.053626 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5088e869-4d40-410e-b52b-f7dd633ba729 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:56.055349 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5088e869-4d40-410e-b52b-f7dd633ba729 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:56.076763 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1397/2795] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:17:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:17:56.078986 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402836c9-c58c-4203-bab1-4deeedfca5bd None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:56.096513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402836c9-c58c-4203-bab1-4deeedfca5bd None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:56.110153 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72019d0-8c96-4ae9-8cc7-6c406e96176e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:56.111845 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72019d0-8c96-4ae9-8cc7-6c406e96176e admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:56.112209 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72019d0-8c96-4ae9-8cc7-6c406e96176e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:56.112769 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72019d0-8c96-4ae9-8cc7-6c406e96176e admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:56.113717 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72019d0-8c96-4ae9-8cc7-6c406e96176e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:56.115864 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72019d0-8c96-4ae9-8cc7-6c406e96176e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:56.229653 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088e869-4d40-410e-b52b-f7dd633ba729 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:56.237850 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1399/2796] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:17:56 2022] POST /identity/v3/projects => generated 403 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:17:56.241875 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088e869-4d40-410e-b52b-f7dd633ba729 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:56.268822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:17:56.312955 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7cf825-f973-48f8-a6a0-3b118ffbd2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:56.313942 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cf825-f973-48f8-a6a0-3b118ffbd2ac admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:56.314440 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cf825-f973-48f8-a6a0-3b118ffbd2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:56.315056 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cf825-f973-48f8-a6a0-3b118ffbd2ac admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:56.316165 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7cf825-f973-48f8-a6a0-3b118ffbd2ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:56.319006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7cf825-f973-48f8-a6a0-3b118ffbd2ac admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:56.443556 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1398/2797] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:17:56 2022] POST /identity/v3/users => generated 302 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:17:56.447649 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72019d0-8c96-4ae9-8cc7-6c406e96176e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:56.459556 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72019d0-8c96-4ae9-8cc7-6c406e96176e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:56.465937 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da8d05a-2c74-4cb0-ba41-b8941ff57cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:56.472337 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da8d05a-2c74-4cb0-ba41-b8941ff57cb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:56.472337 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da8d05a-2c74-4cb0-ba41-b8941ff57cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:56.472337 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da8d05a-2c74-4cb0-ba41-b8941ff57cb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:56.472337 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da8d05a-2c74-4cb0-ba41-b8941ff57cb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:56.472337 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da8d05a-2c74-4cb0-ba41-b8941ff57cb1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:56.488679 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1400/2798] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:17:56.490605 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7cf825-f973-48f8-a6a0-3b118ffbd2ac admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:56.502548 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7cf825-f973-48f8-a6a0-3b118ffbd2ac admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:56.508535 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121667cf-f709-4b83-aaeb-029fd58a76ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:56.509431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121667cf-f709-4b83-aaeb-029fd58a76ce admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:56.509701 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121667cf-f709-4b83-aaeb-029fd58a76ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:56.509985 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121667cf-f709-4b83-aaeb-029fd58a76ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:56.510879 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121667cf-f709-4b83-aaeb-029fd58a76ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:56.513011 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121667cf-f709-4b83-aaeb-029fd58a76ce admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:56.526117 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1399/2799] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:17:56.530970 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da8d05a-2c74-4cb0-ba41-b8941ff57cb1 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:56.540839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da8d05a-2c74-4cb0-ba41-b8941ff57cb1 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:56.546181 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b612594-419a-49a2-9f96-87dfa7b3964c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:56.546807 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b612594-419a-49a2-9f96-87dfa7b3964c admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:56.547110 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b612594-419a-49a2-9f96-87dfa7b3964c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:56.548042 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b612594-419a-49a2-9f96-87dfa7b3964c admin admin] PATH_INFO: `/v3/projects/4bfbc16f15c9478ab61247c1961128f9/users/0d9ed511e9874a3a8209a43e8e254582/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:56.562983 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b612594-419a-49a2-9f96-87dfa7b3964c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bfbc16f15c9478ab61247c1961128f9, user_id=0d9ed511e9874a3a8209a43e8e254582, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:56.565776 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b612594-419a-49a2-9f96-87dfa7b3964c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:56.650198 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1401/2800] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:17:56 2022] PUT /identity/v3/projects/4bfbc16f15c9478ab61247c1961128f9/users/0d9ed511e9874a3a8209a43e8e254582/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:17:56.656030 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e2e1e-c63a-43b0-be3e-58e0fd29843a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:56.656327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e2e1e-c63a-43b0-be3e-58e0fd29843a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:56.656613 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e2e1e-c63a-43b0-be3e-58e0fd29843a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:56.677504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f08e2e1e-c63a-43b0-be3e-58e0fd29843a None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:17:56.680169 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f08e2e1e-c63a-43b0-be3e-58e0fd29843a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:56.775746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1400/2801] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:17:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:17:56.784551 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b612594-419a-49a2-9f96-87dfa7b3964c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:56.786409 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3b612594-419a-49a2-9f96-87dfa7b3964c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:56.887663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b612594-419a-49a2-9f96-87dfa7b3964c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:56.895674 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f09474-2020-4d0e-9cc0-eae6dbea7c7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:56.895674 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f09474-2020-4d0e-9cc0-eae6dbea7c7b service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:56.895674 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f09474-2020-4d0e-9cc0-eae6dbea7c7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:56.895674 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f09474-2020-4d0e-9cc0-eae6dbea7c7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:56.906746 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f09474-2020-4d0e-9cc0-eae6dbea7c7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:56.909527 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f09474-2020-4d0e-9cc0-eae6dbea7c7b service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:56.923030 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1402/2802] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:17:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:17:58.729387 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172e3de-26b3-4728-a926-c4a71af60855 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:58.729890 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172e3de-26b3-4728-a926-c4a71af60855 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:58.730167 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172e3de-26b3-4728-a926-c4a71af60855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:58.840547 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c172e3de-26b3-4728-a926-c4a71af60855 None None] MFA Rules not processed for user `0d9ed511e9874a3a8209a43e8e254582`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:17:58.842599 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c172e3de-26b3-4728-a926-c4a71af60855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:17:58.984517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1401/2803] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:17:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:17:58.988713 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f09474-2020-4d0e-9cc0-eae6dbea7c7b service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:59.004341 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f09474-2020-4d0e-9cc0-eae6dbea7c7b service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:59.019445 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c1298-a05b-421b-b098-07a368ed82e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:59.021259 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c1298-a05b-421b-b098-07a368ed82e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:59.021857 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c1298-a05b-421b-b098-07a368ed82e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:59.022376 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c1298-a05b-421b-b098-07a368ed82e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:59.030016 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c1298-a05b-421b-b098-07a368ed82e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:59.032794 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c1298-a05b-421b-b098-07a368ed82e1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:59.165282 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1403/2804] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:17:58 2022] POST /identity/v3/projects => generated 401 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:17:59.171394 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172e3de-26b3-4728-a926-c4a71af60855 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:59.190235 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172e3de-26b3-4728-a926-c4a71af60855 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:59.231211 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:17:59.268087 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca13d6-cd9e-470f-a196-969e5f0658d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:59.268805 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca13d6-cd9e-470f-a196-969e5f0658d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:59.269095 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca13d6-cd9e-470f-a196-969e5f0658d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:59.269370 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca13d6-cd9e-470f-a196-969e5f0658d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:59.270076 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca13d6-cd9e-470f-a196-969e5f0658d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:59.272113 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca13d6-cd9e-470f-a196-969e5f0658d0 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:59.383209 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1402/2805] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:17:59 2022] POST /identity/v3/users => generated 307 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:17:59.387287 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c1298-a05b-421b-b098-07a368ed82e1 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:59.398523 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c1298-a05b-421b-b098-07a368ed82e1 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:59.405483 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9323c4-0d6b-40ba-a208-7d2ab9ae6a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:59.405483 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9323c4-0d6b-40ba-a208-7d2ab9ae6a2c admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:59.405825 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9323c4-0d6b-40ba-a208-7d2ab9ae6a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:59.406092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9323c4-0d6b-40ba-a208-7d2ab9ae6a2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:59.407310 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9323c4-0d6b-40ba-a208-7d2ab9ae6a2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:59.408963 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9323c4-0d6b-40ba-a208-7d2ab9ae6a2c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:59.425103 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1404/2806] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:17:59.431211 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca13d6-cd9e-470f-a196-969e5f0658d0 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:59.442780 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca13d6-cd9e-470f-a196-969e5f0658d0 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:59.447661 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9734238a-c796-45e5-a399-5b6f35163efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:59.448572 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734238a-c796-45e5-a399-5b6f35163efe admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:59.448859 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734238a-c796-45e5-a399-5b6f35163efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:59.449132 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9734238a-c796-45e5-a399-5b6f35163efe admin admin] PATH_INFO: `/v3/projects/0166907c13bf485bbc2438f4c9c52732/users/1c3fe936c5eb442aa60ff9128632225a/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:59.469319 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9734238a-c796-45e5-a399-5b6f35163efe admin admin] RBAC: Authorizing `identity:create_grant(project_id=0166907c13bf485bbc2438f4c9c52732, user_id=1c3fe936c5eb442aa60ff9128632225a, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:59.472865 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9734238a-c796-45e5-a399-5b6f35163efe admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:59.578139 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1403/2807] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:17:59 2022] PUT /identity/v3/projects/0166907c13bf485bbc2438f4c9c52732/users/1c3fe936c5eb442aa60ff9128632225a/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:17:59.581879 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9323c4-0d6b-40ba-a208-7d2ab9ae6a2c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:59.602454 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9323c4-0d6b-40ba-a208-7d2ab9ae6a2c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:59.647982 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:17:59.686627 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa294e5e-5e89-46fb-b968-ff64519f656a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:59.687568 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa294e5e-5e89-46fb-b968-ff64519f656a admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:59.687872 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa294e5e-5e89-46fb-b968-ff64519f656a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:59.688144 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa294e5e-5e89-46fb-b968-ff64519f656a admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:59.688991 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa294e5e-5e89-46fb-b968-ff64519f656a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:59.690623 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa294e5e-5e89-46fb-b968-ff64519f656a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:59.712072 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1405/2808] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:17:59.713381 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9734238a-c796-45e5-a399-5b6f35163efe admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:17:59.730862 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9734238a-c796-45e5-a399-5b6f35163efe admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:17:59.741925 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f287ce-dfb2-400d-b6c4-46144284a1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:17:59.742836 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f287ce-dfb2-400d-b6c4-46144284a1fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:59.743390 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f287ce-dfb2-400d-b6c4-46144284a1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:59.743672 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f287ce-dfb2-400d-b6c4-46144284a1fb admin admin] PATH_INFO: `/v3/projects/0166907c13bf485bbc2438f4c9c52732/users/1c3fe936c5eb442aa60ff9128632225a/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:59.746051 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f287ce-dfb2-400d-b6c4-46144284a1fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0166907c13bf485bbc2438f4c9c52732, user_id=1c3fe936c5eb442aa60ff9128632225a, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:17:59.749155 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f287ce-dfb2-400d-b6c4-46144284a1fb admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:17:59.762153 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1404/2809] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:17:59 2022] PUT /identity/v3/projects/0166907c13bf485bbc2438f4c9c52732/users/1c3fe936c5eb442aa60ff9128632225a/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:17:59.860522 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416442f9-8d45-44bc-865f-ba319f3d5da9 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:17:59.862660 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416442f9-8d45-44bc-865f-ba319f3d5da9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:17:59.862660 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416442f9-8d45-44bc-865f-ba319f3d5da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:17:59.893228 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-416442f9-8d45-44bc-865f-ba319f3d5da9 None None] MFA Rules not processed for user `2ebdcce97d7c49aa8edaf8d2fb069962`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:17:59.894437 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-416442f9-8d45-44bc-865f-ba319f3d5da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:00.024592 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1406/2810] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:17:59 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:18:00.036288 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f287ce-dfb2-400d-b6c4-46144284a1fb admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:00.038185 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c2f287ce-dfb2-400d-b6c4-46144284a1fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:00.137623 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f287ce-dfb2-400d-b6c4-46144284a1fb admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:00.143661 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee17ee-ca00-4755-a1f0-036524ba7e60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:00.144508 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee17ee-ca00-4755-a1f0-036524ba7e60 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:00.145304 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee17ee-ca00-4755-a1f0-036524ba7e60 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:00.145444 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee17ee-ca00-4755-a1f0-036524ba7e60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:00.147319 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-58ee17ee-ca00-4755-a1f0-036524ba7e60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:00.217701 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee17ee-ca00-4755-a1f0-036524ba7e60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:00.217701 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee17ee-ca00-4755-a1f0-036524ba7e60 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:00.233017 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1405/2811] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:18:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:18:01.092005 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7bdbc-1e61-4c57-af34-c3747ebcd30f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:01.092525 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7bdbc-1e61-4c57-af34-c3747ebcd30f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:01.092674 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7bdbc-1e61-4c57-af34-c3747ebcd30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:01.131906 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c9a7bdbc-1e61-4c57-af34-c3747ebcd30f None None] MFA Rules not processed for user `1c3fe936c5eb442aa60ff9128632225a`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:18:01.133368 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c9a7bdbc-1e61-4c57-af34-c3747ebcd30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:01.214330 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1407/2812] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:18:01 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:18:02.801461 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee17ee-ca00-4755-a1f0-036524ba7e60 service nova] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:02.842347 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:18:02.881747 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee17ee-ca00-4755-a1f0-036524ba7e60 service nova] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:02.886341 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ac64f-7e87-48fe-917c-ce6ea05f891b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:02.887239 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ac64f-7e87-48fe-917c-ce6ea05f891b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:02.887524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ac64f-7e87-48fe-917c-ce6ea05f891b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:02.887858 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ac64f-7e87-48fe-917c-ce6ea05f891b admin admin] PATH_INFO: `/v3/users/0d9ed511e9874a3a8209a43e8e254582` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:02.889698 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ac64f-7e87-48fe-917c-ce6ea05f891b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d9ed511e9874a3a8209a43e8e254582)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:02.891665 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ac64f-7e87-48fe-917c-ce6ea05f891b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:03.125683 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9d4ac64f-7e87-48fe-917c-ce6ea05f891b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d9ed511e9874a3a8209a43e8e254582'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:03.126648 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9d4ac64f-7e87-48fe-917c-ce6ea05f891b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d9ed511e9874a3a8209a43e8e254582'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:03.172381 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9d4ac64f-7e87-48fe-917c-ce6ea05f891b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d9ed511e9874a3a8209a43e8e254582'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:03.182893 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9d4ac64f-7e87-48fe-917c-ce6ea05f891b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d9ed511e9874a3a8209a43e8e254582'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:03.242121 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018dedb9-3fa5-4efd-90db-93f6ea4632d0 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:03.242559 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018dedb9-3fa5-4efd-90db-93f6ea4632d0 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:03.242656 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018dedb9-3fa5-4efd-90db-93f6ea4632d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:03.265615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-018dedb9-3fa5-4efd-90db-93f6ea4632d0 None None] MFA Rules not processed for user `3fa08242da514c0d9fa719830dcfc760`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:18:03.267078 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-018dedb9-3fa5-4efd-90db-93f6ea4632d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:03.276630 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1408/2813] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:18:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:18:03.309492 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9d4ac64f-7e87-48fe-917c-ce6ea05f891b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d9ed511e9874a3a8209a43e8e254582'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:03.320482 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1406/2814] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:18:02 2022] DELETE /identity/v3/users/0d9ed511e9874a3a8209a43e8e254582 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:18:03.585977 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018dedb9-3fa5-4efd-90db-93f6ea4632d0 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:03.611078 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018dedb9-3fa5-4efd-90db-93f6ea4632d0 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:03.659106 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:18:03.692923 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc8e55-a68e-4d18-b0f3-04d534e03ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:03.694041 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc8e55-a68e-4d18-b0f3-04d534e03ba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:03.694658 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc8e55-a68e-4d18-b0f3-04d534e03ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:03.695092 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc8e55-a68e-4d18-b0f3-04d534e03ba6 admin admin] PATH_INFO: `/v3/projects/4bfbc16f15c9478ab61247c1961128f9` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:03.696556 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc8e55-a68e-4d18-b0f3-04d534e03ba6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bfbc16f15c9478ab61247c1961128f9)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:03.698513 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc8e55-a68e-4d18-b0f3-04d534e03ba6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:03.853014 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-19cc8e55-a68e-4d18-b0f3-04d534e03ba6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bfbc16f15c9478ab61247c1961128f9', 'request_id': 'req-19cc8e55-a68e-4d18-b0f3-04d534e03ba6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:03.925822 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-19cc8e55-a68e-4d18-b0f3-04d534e03ba6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bfbc16f15c9478ab61247c1961128f9', 'request_id': 'req-19cc8e55-a68e-4d18-b0f3-04d534e03ba6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:03.932134 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-19cc8e55-a68e-4d18-b0f3-04d534e03ba6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bfbc16f15c9478ab61247c1961128f9', 'request_id': 'req-19cc8e55-a68e-4d18-b0f3-04d534e03ba6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:03.944511 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-19cc8e55-a68e-4d18-b0f3-04d534e03ba6 admin admin] The token cache is being invalidate because project 4bfbc16f15c9478ab61247c1961128f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:18:03.944804 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-19cc8e55-a68e-4d18-b0f3-04d534e03ba6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:03.945479 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-19cc8e55-a68e-4d18-b0f3-04d534e03ba6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:03.947362 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1409/2815] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:18:03 2022] DELETE /identity/v3/projects/4bfbc16f15c9478ab61247c1961128f9 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:18:03.951745 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ac64f-7e87-48fe-917c-ce6ea05f891b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:03.953210 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9d4ac64f-7e87-48fe-917c-ce6ea05f891b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:03.966800 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ac64f-7e87-48fe-917c-ce6ea05f891b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:04.037031 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cdbe37-8e1b-4daa-b0ac-180dd68c488f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:04.038069 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cdbe37-8e1b-4daa-b0ac-180dd68c488f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:04.038382 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cdbe37-8e1b-4daa-b0ac-180dd68c488f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:04.038660 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cdbe37-8e1b-4daa-b0ac-180dd68c488f admin admin] PATH_INFO: `/v3/users/1c3fe936c5eb442aa60ff9128632225a` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:04.040112 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cdbe37-8e1b-4daa-b0ac-180dd68c488f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c3fe936c5eb442aa60ff9128632225a)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:04.042150 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cdbe37-8e1b-4daa-b0ac-180dd68c488f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:04.252006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:18:04.267824 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-35cdbe37-8e1b-4daa-b0ac-180dd68c488f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c3fe936c5eb442aa60ff9128632225a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:04.269824 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-35cdbe37-8e1b-4daa-b0ac-180dd68c488f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c3fe936c5eb442aa60ff9128632225a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:04.308466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-35cdbe37-8e1b-4daa-b0ac-180dd68c488f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c3fe936c5eb442aa60ff9128632225a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:04.330018 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-35cdbe37-8e1b-4daa-b0ac-180dd68c488f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c3fe936c5eb442aa60ff9128632225a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:04.446094 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-35cdbe37-8e1b-4daa-b0ac-180dd68c488f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c3fe936c5eb442aa60ff9128632225a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:04.456667 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1407/2816] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:18:03 2022] DELETE /identity/v3/users/1c3fe936c5eb442aa60ff9128632225a => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:18:04.721948 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc8e55-a68e-4d18-b0f3-04d534e03ba6 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:04.723737 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-19cc8e55-a68e-4d18-b0f3-04d534e03ba6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:04.743146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc8e55-a68e-4d18-b0f3-04d534e03ba6 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:04.826348 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4a111-bf9f-45ea-9ab6-4fdb537fd092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:04.827676 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4a111-bf9f-45ea-9ab6-4fdb537fd092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:04.828084 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4a111-bf9f-45ea-9ab6-4fdb537fd092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:04.828505 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4a111-bf9f-45ea-9ab6-4fdb537fd092 admin admin] PATH_INFO: `/v3/projects/0166907c13bf485bbc2438f4c9c52732` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:04.829729 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb4a111-bf9f-45ea-9ab6-4fdb537fd092 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0166907c13bf485bbc2438f4c9c52732)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:04.831681 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb4a111-bf9f-45ea-9ab6-4fdb537fd092 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:04.967508 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1cb4a111-bf9f-45ea-9ab6-4fdb537fd092 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0166907c13bf485bbc2438f4c9c52732', 'request_id': 'req-1cb4a111-bf9f-45ea-9ab6-4fdb537fd092'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:05.029401 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1cb4a111-bf9f-45ea-9ab6-4fdb537fd092 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0166907c13bf485bbc2438f4c9c52732', 'request_id': 'req-1cb4a111-bf9f-45ea-9ab6-4fdb537fd092'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:05.035233 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:18:05.036663 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1cb4a111-bf9f-45ea-9ab6-4fdb537fd092 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0166907c13bf485bbc2438f4c9c52732', 'request_id': 'req-1cb4a111-bf9f-45ea-9ab6-4fdb537fd092'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:05.048433 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1cb4a111-bf9f-45ea-9ab6-4fdb537fd092 admin admin] The token cache is being invalidate because project 0166907c13bf485bbc2438f4c9c52732 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:18:05.052348 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1cb4a111-bf9f-45ea-9ab6-4fdb537fd092 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:05.052348 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1cb4a111-bf9f-45ea-9ab6-4fdb537fd092 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:05.052348 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1410/2817] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:18:04 2022] DELETE /identity/v3/projects/0166907c13bf485bbc2438f4c9c52732 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:18:05.055821 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca93512-5512-4527-8777-e4872fe90a80 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:05.056292 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca93512-5512-4527-8777-e4872fe90a80 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:05.056431 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca93512-5512-4527-8777-e4872fe90a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:05.075356 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4ca93512-5512-4527-8777-e4872fe90a80 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:18:05.077140 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4ca93512-5512-4527-8777-e4872fe90a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:05.140981 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1408/2818] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:18:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:18:05.150261 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5925e-d5d2-4934-863a-3f3edb2f7f6a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:05.150885 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5925e-d5d2-4934-863a-3f3edb2f7f6a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:05.151446 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5925e-d5d2-4934-863a-3f3edb2f7f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:05.173741 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-34e5925e-d5d2-4934-863a-3f3edb2f7f6a None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:18:05.175010 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-34e5925e-d5d2-4934-863a-3f3edb2f7f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:05.183199 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1411/2819] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:18:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:18:05.188047 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca93512-5512-4527-8777-e4872fe90a80 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:05.198700 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca93512-5512-4527-8777-e4872fe90a80 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:05.203326 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543cb48-8dd2-401f-8e56-7f4c7ade4c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:05.203988 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543cb48-8dd2-401f-8e56-7f4c7ade4c39 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:05.204271 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543cb48-8dd2-401f-8e56-7f4c7ade4c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:05.204578 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543cb48-8dd2-401f-8e56-7f4c7ade4c39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:05.205306 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8543cb48-8dd2-401f-8e56-7f4c7ade4c39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:18:05.205614 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8543cb48-8dd2-401f-8e56-7f4c7ade4c39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:05.207226 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8543cb48-8dd2-401f-8e56-7f4c7ade4c39 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:05.219524 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1409/2820] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:18:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:18:05.229514 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5925e-d5d2-4934-863a-3f3edb2f7f6a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:05.234760 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5925e-d5d2-4934-863a-3f3edb2f7f6a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:05.244635 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089bb7d6-fd43-44d0-8de1-be1825e0d148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:05.245204 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089bb7d6-fd43-44d0-8de1-be1825e0d148 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:05.245485 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089bb7d6-fd43-44d0-8de1-be1825e0d148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:05.245756 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089bb7d6-fd43-44d0-8de1-be1825e0d148 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:05.246449 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089bb7d6-fd43-44d0-8de1-be1825e0d148 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:05.249186 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089bb7d6-fd43-44d0-8de1-be1825e0d148 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:05.339880 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1412/2821] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:18:05 2022] POST /identity/v3/projects => generated 355 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:18:05.344482 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543cb48-8dd2-401f-8e56-7f4c7ade4c39 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:05.354601 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543cb48-8dd2-401f-8e56-7f4c7ade4c39 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:05.420883 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86619bb2-a065-48e7-9d20-79417b46a8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:05.421555 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86619bb2-a065-48e7-9d20-79417b46a8c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:05.423626 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86619bb2-a065-48e7-9d20-79417b46a8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:05.423626 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86619bb2-a065-48e7-9d20-79417b46a8c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:05.423626 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86619bb2-a065-48e7-9d20-79417b46a8c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:05.424961 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86619bb2-a065-48e7-9d20-79417b46a8c8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:05.519368 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1410/2822] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:18:05 2022] POST /identity/v3/users => generated 278 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:18:05.520542 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089bb7d6-fd43-44d0-8de1-be1825e0d148 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:05.531431 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089bb7d6-fd43-44d0-8de1-be1825e0d148 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:05.536064 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d56328-fedf-4c52-be34-bd3dafaead70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:05.536774 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d56328-fedf-4c52-be34-bd3dafaead70 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:05.537161 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d56328-fedf-4c52-be34-bd3dafaead70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:05.537438 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d56328-fedf-4c52-be34-bd3dafaead70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:05.538163 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d56328-fedf-4c52-be34-bd3dafaead70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:05.539770 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d56328-fedf-4c52-be34-bd3dafaead70 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:05.555200 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1413/2823] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:18:05.561695 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86619bb2-a065-48e7-9d20-79417b46a8c8 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:05.567309 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86619bb2-a065-48e7-9d20-79417b46a8c8 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:05.572316 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec336fa-9e0b-432b-88ca-0612ae20447e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:05.572999 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec336fa-9e0b-432b-88ca-0612ae20447e admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:05.573280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec336fa-9e0b-432b-88ca-0612ae20447e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:05.573685 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec336fa-9e0b-432b-88ca-0612ae20447e admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:05.574417 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec336fa-9e0b-432b-88ca-0612ae20447e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:05.576166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec336fa-9e0b-432b-88ca-0612ae20447e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:05.587718 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1411/2824] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:18:05.593341 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d56328-fedf-4c52-be34-bd3dafaead70 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:05.603326 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d56328-fedf-4c52-be34-bd3dafaead70 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:05.608389 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24034f4-576b-498f-919c-836f398d085d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:05.609254 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24034f4-576b-498f-919c-836f398d085d admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:05.609531 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24034f4-576b-498f-919c-836f398d085d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:05.609802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24034f4-576b-498f-919c-836f398d085d admin admin] PATH_INFO: `/v3/projects/6bbf17fe16044b1888e2cdfa21e3b0a0/users/2d963d2b34fc494e9d951d096adc5c9d/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:05.624218 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24034f4-576b-498f-919c-836f398d085d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bbf17fe16044b1888e2cdfa21e3b0a0, user_id=2d963d2b34fc494e9d951d096adc5c9d, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:05.627080 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24034f4-576b-498f-919c-836f398d085d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:05.692866 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1414/2825] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:18:05 2022] PUT /identity/v3/projects/6bbf17fe16044b1888e2cdfa21e3b0a0/users/2d963d2b34fc494e9d951d096adc5c9d/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:18:05.704689 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d496e-e042-403f-9607-1f7adc4d5dbb None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:05.705000 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d496e-e042-403f-9607-1f7adc4d5dbb None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:05.705329 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d496e-e042-403f-9607-1f7adc4d5dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:05.733117 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d14d496e-e042-403f-9607-1f7adc4d5dbb None None] MFA Rules not processed for user `2d963d2b34fc494e9d951d096adc5c9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:18:05.734832 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d14d496e-e042-403f-9607-1f7adc4d5dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:05.751380 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:18:05.806616 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1412/2826] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:18:05 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:18:05.811712 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24034f4-576b-498f-919c-836f398d085d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:05.885574 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24034f4-576b-498f-919c-836f398d085d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:05.890399 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3a208-d617-4161-ab18-9004b1b6b0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:05.891091 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3a208-d617-4161-ab18-9004b1b6b0c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:05.891402 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3a208-d617-4161-ab18-9004b1b6b0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:05.891651 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3a208-d617-4161-ab18-9004b1b6b0c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:05.892438 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a3a208-d617-4161-ab18-9004b1b6b0c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:05.894358 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a3a208-d617-4161-ab18-9004b1b6b0c5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:05.968850 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1415/2827] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:18:05 2022] POST /identity/v3/projects => generated 357 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:18:05.972888 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d496e-e042-403f-9607-1f7adc4d5dbb None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:05.984996 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d496e-e042-403f-9607-1f7adc4d5dbb None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:06.044131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb3791-2194-4ec7-b12c-a28ce1a7eb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:06.045376 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb3791-2194-4ec7-b12c-a28ce1a7eb44 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:06.045661 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb3791-2194-4ec7-b12c-a28ce1a7eb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:06.045934 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb3791-2194-4ec7-b12c-a28ce1a7eb44 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:06.046631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb3791-2194-4ec7-b12c-a28ce1a7eb44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:06.048575 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb3791-2194-4ec7-b12c-a28ce1a7eb44 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:06.125742 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1413/2828] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:18:05 2022] POST /identity/v3/users => generated 285 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:18:06.129770 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3a208-d617-4161-ab18-9004b1b6b0c5 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:06.140087 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3a208-d617-4161-ab18-9004b1b6b0c5 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:06.144818 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c716b-3de5-451e-b286-2aea6e202333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:06.145467 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c716b-3de5-451e-b286-2aea6e202333 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:06.145754 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c716b-3de5-451e-b286-2aea6e202333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:06.146027 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c716b-3de5-451e-b286-2aea6e202333 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:06.146742 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c716b-3de5-451e-b286-2aea6e202333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:06.148370 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c716b-3de5-451e-b286-2aea6e202333 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:06.159217 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1416/2829] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:18:06.163304 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb3791-2194-4ec7-b12c-a28ce1a7eb44 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:06.175329 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb3791-2194-4ec7-b12c-a28ce1a7eb44 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:06.180825 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb4cd6-9134-4446-88fb-a2f3b760ccd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:06.181846 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb4cd6-9134-4446-88fb-a2f3b760ccd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:06.182131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb4cd6-9134-4446-88fb-a2f3b760ccd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:06.182412 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb4cd6-9134-4446-88fb-a2f3b760ccd2 admin admin] PATH_INFO: `/v3/projects/6d40902a5c9743e3a98d0b51a3dce287/users/4417ebde31d24133877f07bf24cf4c75/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:06.197931 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbb4cd6-9134-4446-88fb-a2f3b760ccd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d40902a5c9743e3a98d0b51a3dce287, user_id=4417ebde31d24133877f07bf24cf4c75, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:06.200706 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbb4cd6-9134-4446-88fb-a2f3b760ccd2 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:06.292053 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1414/2830] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:18:06 2022] PUT /identity/v3/projects/6d40902a5c9743e3a98d0b51a3dce287/users/4417ebde31d24133877f07bf24cf4c75/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:18:06.296618 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c716b-3de5-451e-b286-2aea6e202333 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:06.306893 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c716b-3de5-451e-b286-2aea6e202333 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:06.366008 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7348e-ae08-4fb8-bd6f-0b9f4a5a11c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:06.366672 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7348e-ae08-4fb8-bd6f-0b9f4a5a11c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:06.366957 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7348e-ae08-4fb8-bd6f-0b9f4a5a11c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:06.367237 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7348e-ae08-4fb8-bd6f-0b9f4a5a11c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:06.367980 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d7348e-ae08-4fb8-bd6f-0b9f4a5a11c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:06.369628 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d7348e-ae08-4fb8-bd6f-0b9f4a5a11c5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:06.371796 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:18:06.380921 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1417/2831] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:18:06 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:18:06.385107 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb4cd6-9134-4446-88fb-a2f3b760ccd2 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:06.395962 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb4cd6-9134-4446-88fb-a2f3b760ccd2 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:06.400818 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea1d83-84c0-4085-af2b-7731d55254cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:06.401687 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea1d83-84c0-4085-af2b-7731d55254cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:06.401975 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea1d83-84c0-4085-af2b-7731d55254cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:06.402253 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea1d83-84c0-4085-af2b-7731d55254cf admin admin] PATH_INFO: `/v3/projects/6d40902a5c9743e3a98d0b51a3dce287/users/4417ebde31d24133877f07bf24cf4c75/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:06.404819 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea1d83-84c0-4085-af2b-7731d55254cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d40902a5c9743e3a98d0b51a3dce287, user_id=4417ebde31d24133877f07bf24cf4c75, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:06.407467 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea1d83-84c0-4085-af2b-7731d55254cf admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:06.418095 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1415/2832] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:18:06 2022] PUT /identity/v3/projects/6d40902a5c9743e3a98d0b51a3dce287/users/4417ebde31d24133877f07bf24cf4c75/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:18:06.428645 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe049c3-7ca8-4cf1-925a-ea9484b2fb3b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:06.428961 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe049c3-7ca8-4cf1-925a-ea9484b2fb3b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:06.429240 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe049c3-7ca8-4cf1-925a-ea9484b2fb3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:06.456124 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-2fe049c3-7ca8-4cf1-925a-ea9484b2fb3b None None] MFA Rules not processed for user `4417ebde31d24133877f07bf24cf4c75`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:18:06.457357 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2fe049c3-7ca8-4cf1-925a-ea9484b2fb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:06.528861 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1418/2833] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:18:06 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:18:06.539970 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea1d83-84c0-4085-af2b-7731d55254cf admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:06.544811 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-cdea1d83-84c0-4085-af2b-7731d55254cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:06.603737 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea1d83-84c0-4085-af2b-7731d55254cf admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:06.608600 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01eb18-9150-40c3-a25b-a615c7a82297 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:06.609253 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01eb18-9150-40c3-a25b-a615c7a82297 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:06.609533 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01eb18-9150-40c3-a25b-a615c7a82297 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:06.609808 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f01eb18-9150-40c3-a25b-a615c7a82297 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:06.619388 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f01eb18-9150-40c3-a25b-a615c7a82297 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:06.620906 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f01eb18-9150-40c3-a25b-a615c7a82297 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:06.630907 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1416/2834] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:18:06 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:18:06.915833 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe049c3-7ca8-4cf1-925a-ea9484b2fb3b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:06.929162 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe049c3-7ca8-4cf1-925a-ea9484b2fb3b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:06.933127 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052bdce-4ed5-46fe-a8e0-741c5b9e9e75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:06.933878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052bdce-4ed5-46fe-a8e0-741c5b9e9e75 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:06.934211 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052bdce-4ed5-46fe-a8e0-741c5b9e9e75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:06.934426 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052bdce-4ed5-46fe-a8e0-741c5b9e9e75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:06.994125 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7052bdce-4ed5-46fe-a8e0-741c5b9e9e75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:06.995571 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7052bdce-4ed5-46fe-a8e0-741c5b9e9e75 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:07.005097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1419/2835] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:18:06 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:18:16.626716 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=91,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:18:17.001071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=63 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:18:25.565536 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01eb18-9150-40c3-a25b-a615c7a82297 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:25.567096 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2f01eb18-9150-40c3-a25b-a615c7a82297 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:25.632654 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f01eb18-9150-40c3-a25b-a615c7a82297 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:25.636465 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76267ef3-4315-4245-b078-dc5e1400e2ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:25.637153 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76267ef3-4315-4245-b078-dc5e1400e2ec service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:25.637458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76267ef3-4315-4245-b078-dc5e1400e2ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:25.637770 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76267ef3-4315-4245-b078-dc5e1400e2ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:25.639662 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-76267ef3-4315-4245-b078-dc5e1400e2ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:25.649931 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76267ef3-4315-4245-b078-dc5e1400e2ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:25.651359 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76267ef3-4315-4245-b078-dc5e1400e2ec service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:25.659980 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1417/2836] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:18:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:18:28.604027 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052bdce-4ed5-46fe-a8e0-741c5b9e9e75 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:28.663979 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052bdce-4ed5-46fe-a8e0-741c5b9e9e75 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:28.668714 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb7010-9fc8-4b59-a465-f576943c6447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:28.669577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb7010-9fc8-4b59-a465-f576943c6447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:28.669863 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb7010-9fc8-4b59-a465-f576943c6447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:28.670131 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb7010-9fc8-4b59-a465-f576943c6447 admin admin] PATH_INFO: `/v3/users/2d963d2b34fc494e9d951d096adc5c9d` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:28.671321 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb7010-9fc8-4b59-a465-f576943c6447 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d963d2b34fc494e9d951d096adc5c9d)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:28.673296 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb7010-9fc8-4b59-a465-f576943c6447 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:28.867097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-02bb7010-9fc8-4b59-a465-f576943c6447 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d963d2b34fc494e9d951d096adc5c9d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:28.867829 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-02bb7010-9fc8-4b59-a465-f576943c6447 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d963d2b34fc494e9d951d096adc5c9d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:28.927781 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-02bb7010-9fc8-4b59-a465-f576943c6447 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d963d2b34fc494e9d951d096adc5c9d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:28.928576 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-02bb7010-9fc8-4b59-a465-f576943c6447 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d963d2b34fc494e9d951d096adc5c9d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:28.942371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-02bb7010-9fc8-4b59-a465-f576943c6447 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d963d2b34fc494e9d951d096adc5c9d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:28.956016 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1420/2837] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:18:28 2022] DELETE /identity/v3/users/2d963d2b34fc494e9d951d096adc5c9d => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:18:28.960371 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14c355-35e9-43cc-86bc-ea0165669832 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:28.961033 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14c355-35e9-43cc-86bc-ea0165669832 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:28.961613 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de14c355-35e9-43cc-86bc-ea0165669832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:28.981324 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-de14c355-35e9-43cc-86bc-ea0165669832 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:18:28.982784 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-de14c355-35e9-43cc-86bc-ea0165669832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:29.047135 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1418/2838] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:18:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:18:29.059293 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb7010-9fc8-4b59-a465-f576943c6447 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:29.063851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-02bb7010-9fc8-4b59-a465-f576943c6447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:29.137802 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb7010-9fc8-4b59-a465-f576943c6447 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:29.141641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3a2a2-7e3c-4725-8894-66dad830780e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:29.142395 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3a2a2-7e3c-4725-8894-66dad830780e service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:29.142731 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3a2a2-7e3c-4725-8894-66dad830780e service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:29.143064 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3a2a2-7e3c-4725-8894-66dad830780e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:29.153296 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3a2a2-7e3c-4725-8894-66dad830780e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:29.154742 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3a2a2-7e3c-4725-8894-66dad830780e service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:29.164912 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1421/2839] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:18:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:18:29.545015 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14c355-35e9-43cc-86bc-ea0165669832 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:29.546674 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-de14c355-35e9-43cc-86bc-ea0165669832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:29.564089 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de14c355-35e9-43cc-86bc-ea0165669832 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:29.570212 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d37107-1d4c-47cf-90f1-f62b99e7a31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:29.571154 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d37107-1d4c-47cf-90f1-f62b99e7a31e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:29.571433 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d37107-1d4c-47cf-90f1-f62b99e7a31e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:29.571714 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d37107-1d4c-47cf-90f1-f62b99e7a31e admin admin] PATH_INFO: `/v3/projects/6bbf17fe16044b1888e2cdfa21e3b0a0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:29.573587 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d37107-1d4c-47cf-90f1-f62b99e7a31e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bbf17fe16044b1888e2cdfa21e3b0a0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:29.575269 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d37107-1d4c-47cf-90f1-f62b99e7a31e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:29.705308 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-05d37107-1d4c-47cf-90f1-f62b99e7a31e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bbf17fe16044b1888e2cdfa21e3b0a0', 'request_id': 'req-05d37107-1d4c-47cf-90f1-f62b99e7a31e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:29.708606 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-05d37107-1d4c-47cf-90f1-f62b99e7a31e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bbf17fe16044b1888e2cdfa21e3b0a0', 'request_id': 'req-05d37107-1d4c-47cf-90f1-f62b99e7a31e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:29.715766 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-05d37107-1d4c-47cf-90f1-f62b99e7a31e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bbf17fe16044b1888e2cdfa21e3b0a0', 'request_id': 'req-05d37107-1d4c-47cf-90f1-f62b99e7a31e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:29.788886 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-05d37107-1d4c-47cf-90f1-f62b99e7a31e admin admin] The token cache is being invalidate because project 6bbf17fe16044b1888e2cdfa21e3b0a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:18:29.789177 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-05d37107-1d4c-47cf-90f1-f62b99e7a31e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:29.789953 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-05d37107-1d4c-47cf-90f1-f62b99e7a31e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:29.791776 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1419/2840] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:18:29 2022] DELETE /identity/v3/projects/6bbf17fe16044b1888e2cdfa21e3b0a0 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:18:29.795859 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3a2a2-7e3c-4725-8894-66dad830780e service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:29.806055 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3a2a2-7e3c-4725-8894-66dad830780e service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:29.826074 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:18:29.869928 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455e018-6b43-441d-b734-07f14c1b1714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:29.869928 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455e018-6b43-441d-b734-07f14c1b1714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:29.869928 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455e018-6b43-441d-b734-07f14c1b1714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:29.870844 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455e018-6b43-441d-b734-07f14c1b1714 admin admin] PATH_INFO: `/v3/users/4417ebde31d24133877f07bf24cf4c75` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:29.871417 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9455e018-6b43-441d-b734-07f14c1b1714 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4417ebde31d24133877f07bf24cf4c75)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:29.873811 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9455e018-6b43-441d-b734-07f14c1b1714 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:30.049599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9455e018-6b43-441d-b734-07f14c1b1714 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4417ebde31d24133877f07bf24cf4c75'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:30.050307 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9455e018-6b43-441d-b734-07f14c1b1714 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4417ebde31d24133877f07bf24cf4c75'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:30.132040 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9455e018-6b43-441d-b734-07f14c1b1714 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4417ebde31d24133877f07bf24cf4c75'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:30.132824 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9455e018-6b43-441d-b734-07f14c1b1714 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4417ebde31d24133877f07bf24cf4c75'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:30.145233 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9455e018-6b43-441d-b734-07f14c1b1714 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4417ebde31d24133877f07bf24cf4c75'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:30.158869 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1422/2841] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:18:29 2022] DELETE /identity/v3/users/4417ebde31d24133877f07bf24cf4c75 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:18:30.712457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d37107-1d4c-47cf-90f1-f62b99e7a31e admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:30.714005 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-05d37107-1d4c-47cf-90f1-f62b99e7a31e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:30.733769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d37107-1d4c-47cf-90f1-f62b99e7a31e admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:30.761816 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:18:30.795083 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a52c5d-7dd6-4d3c-a9b2-2dffd8299d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:30.795952 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a52c5d-7dd6-4d3c-a9b2-2dffd8299d9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:30.796236 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a52c5d-7dd6-4d3c-a9b2-2dffd8299d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:30.796557 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a52c5d-7dd6-4d3c-a9b2-2dffd8299d9e admin admin] PATH_INFO: `/v3/projects/6d40902a5c9743e3a98d0b51a3dce287` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:30.797781 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a52c5d-7dd6-4d3c-a9b2-2dffd8299d9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d40902a5c9743e3a98d0b51a3dce287)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:30.799964 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a52c5d-7dd6-4d3c-a9b2-2dffd8299d9e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:30.926163 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-70a52c5d-7dd6-4d3c-a9b2-2dffd8299d9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d40902a5c9743e3a98d0b51a3dce287', 'request_id': 'req-70a52c5d-7dd6-4d3c-a9b2-2dffd8299d9e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:30.933066 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-70a52c5d-7dd6-4d3c-a9b2-2dffd8299d9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d40902a5c9743e3a98d0b51a3dce287', 'request_id': 'req-70a52c5d-7dd6-4d3c-a9b2-2dffd8299d9e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:30.938676 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-70a52c5d-7dd6-4d3c-a9b2-2dffd8299d9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d40902a5c9743e3a98d0b51a3dce287', 'request_id': 'req-70a52c5d-7dd6-4d3c-a9b2-2dffd8299d9e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:31.016911 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-70a52c5d-7dd6-4d3c-a9b2-2dffd8299d9e admin admin] The token cache is being invalidate because project 6d40902a5c9743e3a98d0b51a3dce287 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:18:31.017275 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-70a52c5d-7dd6-4d3c-a9b2-2dffd8299d9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:31.018006 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-70a52c5d-7dd6-4d3c-a9b2-2dffd8299d9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:31.019619 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1420/2842] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:18:30 2022] DELETE /identity/v3/projects/6d40902a5c9743e3a98d0b51a3dce287 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:18:31.025282 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71869ee-4df4-43da-9f52-2d288564bf96 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:31.025819 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71869ee-4df4-43da-9f52-2d288564bf96 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:31.025819 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71869ee-4df4-43da-9f52-2d288564bf96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:31.044031 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e71869ee-4df4-43da-9f52-2d288564bf96 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:18:31.045270 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e71869ee-4df4-43da-9f52-2d288564bf96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:31.105984 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1423/2843] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:18:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:18:31.113551 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c905289-35c5-432a-9f7d-26141e827b02 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:31.114050 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c905289-35c5-432a-9f7d-26141e827b02 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:31.114345 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c905289-35c5-432a-9f7d-26141e827b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:31.140283 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-3c905289-35c5-432a-9f7d-26141e827b02 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:18:31.141449 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3c905289-35c5-432a-9f7d-26141e827b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:31.165938 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1421/2844] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:18:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:18:31.175203 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71869ee-4df4-43da-9f52-2d288564bf96 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:31.186112 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71869ee-4df4-43da-9f52-2d288564bf96 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:31.191057 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fab821-d724-4e2e-9037-e7e773d596f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:31.192012 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fab821-d724-4e2e-9037-e7e773d596f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:31.192422 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fab821-d724-4e2e-9037-e7e773d596f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:31.197425 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fab821-d724-4e2e-9037-e7e773d596f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:31.197425 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fab821-d724-4e2e-9037-e7e773d596f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:18:31.197425 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fab821-d724-4e2e-9037-e7e773d596f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:31.197425 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fab821-d724-4e2e-9037-e7e773d596f8 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:31.208950 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1424/2845] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 20:18:31.213523 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c905289-35c5-432a-9f7d-26141e827b02 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:31.225376 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c905289-35c5-432a-9f7d-26141e827b02 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:31.240458 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626282ef-0069-442c-bded-55261f3c83dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:31.241140 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626282ef-0069-442c-bded-55261f3c83dc admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:31.241420 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626282ef-0069-442c-bded-55261f3c83dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:31.241689 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626282ef-0069-442c-bded-55261f3c83dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:31.242394 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626282ef-0069-442c-bded-55261f3c83dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:31.244436 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626282ef-0069-442c-bded-55261f3c83dc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:31.330421 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1422/2846] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:18:31 2022] POST /identity/v3/projects => generated 367 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:18:31.332055 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fab821-d724-4e2e-9037-e7e773d596f8 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:31.343274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fab821-d724-4e2e-9037-e7e773d596f8 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:31.374427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:18:31.405556 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c966d68d-91e0-4b13-b610-8ad1713104de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:31.406226 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966d68d-91e0-4b13-b610-8ad1713104de admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:31.406506 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966d68d-91e0-4b13-b610-8ad1713104de admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:31.406778 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966d68d-91e0-4b13-b610-8ad1713104de admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:31.407476 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c966d68d-91e0-4b13-b610-8ad1713104de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:31.409443 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c966d68d-91e0-4b13-b610-8ad1713104de admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:31.541918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1425/2847] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:18:31 2022] POST /identity/v3/users => generated 284 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 20:18:31.542503 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626282ef-0069-442c-bded-55261f3c83dc admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:31.553283 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626282ef-0069-442c-bded-55261f3c83dc admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:31.558112 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c1259-edeb-4e61-b6ab-ede1615b638c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:31.558852 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c1259-edeb-4e61-b6ab-ede1615b638c admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:31.559204 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c1259-edeb-4e61-b6ab-ede1615b638c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:31.559547 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c1259-edeb-4e61-b6ab-ede1615b638c admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:31.560379 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3c1259-edeb-4e61-b6ab-ede1615b638c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:31.562064 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3c1259-edeb-4e61-b6ab-ede1615b638c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:31.574873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1423/2848] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:18:31.578171 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c966d68d-91e0-4b13-b610-8ad1713104de admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:31.589392 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c966d68d-91e0-4b13-b610-8ad1713104de admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:31.594702 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae88ed2-4ca7-4699-a36d-254f4c0a9740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:31.595363 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae88ed2-4ca7-4699-a36d-254f4c0a9740 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:31.595643 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae88ed2-4ca7-4699-a36d-254f4c0a9740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:31.596039 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae88ed2-4ca7-4699-a36d-254f4c0a9740 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:31.596797 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae88ed2-4ca7-4699-a36d-254f4c0a9740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:31.598401 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae88ed2-4ca7-4699-a36d-254f4c0a9740 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:31.609077 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1426/2849] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:18:31.613254 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c1259-edeb-4e61-b6ab-ede1615b638c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:31.624523 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c1259-edeb-4e61-b6ab-ede1615b638c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:31.630745 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f59f6-f2bc-4f2d-99e9-cdd1f94348d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:31.634202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f59f6-f2bc-4f2d-99e9-cdd1f94348d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:31.634202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f59f6-f2bc-4f2d-99e9-cdd1f94348d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:31.634202 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f59f6-f2bc-4f2d-99e9-cdd1f94348d1 admin admin] PATH_INFO: `/v3/projects/22dcfc78ca3642b683ff276d0caa0eba/users/2b71c28e501c4de0a947cbf4a05ce89b/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:31.648067 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f59f6-f2bc-4f2d-99e9-cdd1f94348d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22dcfc78ca3642b683ff276d0caa0eba, user_id=2b71c28e501c4de0a947cbf4a05ce89b, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:31.650984 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f59f6-f2bc-4f2d-99e9-cdd1f94348d1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:31.758892 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1424/2850] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:18:31 2022] PUT /identity/v3/projects/22dcfc78ca3642b683ff276d0caa0eba/users/2b71c28e501c4de0a947cbf4a05ce89b/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:18:31.769796 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ce724-56c1-4c30-bea5-006a6d897c49 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:31.770222 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ce724-56c1-4c30-bea5-006a6d897c49 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:31.770694 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ce724-56c1-4c30-bea5-006a6d897c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:31.801676 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e62ce724-56c1-4c30-bea5-006a6d897c49 None None] MFA Rules not processed for user `2b71c28e501c4de0a947cbf4a05ce89b`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:18:31.803543 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e62ce724-56c1-4c30-bea5-006a6d897c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:31.874301 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1427/2851] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:18:31 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:18:31.878854 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f59f6-f2bc-4f2d-99e9-cdd1f94348d1 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:31.930644 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:18:31.940516 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f59f6-f2bc-4f2d-99e9-cdd1f94348d1 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:31.945649 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056c269-528a-462f-9acb-49cfbdb5f898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:31.946298 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056c269-528a-462f-9acb-49cfbdb5f898 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:31.946580 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056c269-528a-462f-9acb-49cfbdb5f898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:31.946858 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056c269-528a-462f-9acb-49cfbdb5f898 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:31.947544 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056c269-528a-462f-9acb-49cfbdb5f898 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:31.949469 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056c269-528a-462f-9acb-49cfbdb5f898 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:31.977666 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ce724-56c1-4c30-bea5-006a6d897c49 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:31.979108 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e62ce724-56c1-4c30-bea5-006a6d897c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:31.993908 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ce724-56c1-4c30-bea5-006a6d897c49 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:31.998532 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c0066-2133-4a9f-bece-c53d15f4bcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:32.002839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c0066-2133-4a9f-bece-c53d15f4bcaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:32.002839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c0066-2133-4a9f-bece-c53d15f4bcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:32.002839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c0066-2133-4a9f-bece-c53d15f4bcaa admin admin] PATH_INFO: `/v3/users/1b02048ea85d46889e86bb8d073bf456` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:32.002839 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7c0066-2133-4a9f-bece-c53d15f4bcaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b02048ea85d46889e86bb8d073bf456)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:32.003272 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7c0066-2133-4a9f-bece-c53d15f4bcaa admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:32.029305 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1425/2852] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:18:31 2022] POST /identity/v3/projects => generated 367 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:18:32.033122 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056c269-528a-462f-9acb-49cfbdb5f898 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:32.093775 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056c269-528a-462f-9acb-49cfbdb5f898 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:32.099898 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c823fa6-1218-4b03-81a8-8b09c106e16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:32.100597 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c823fa6-1218-4b03-81a8-8b09c106e16f admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:32.106215 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c823fa6-1218-4b03-81a8-8b09c106e16f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:32.106215 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c823fa6-1218-4b03-81a8-8b09c106e16f admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:32.106215 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c823fa6-1218-4b03-81a8-8b09c106e16f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:32.106215 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c823fa6-1218-4b03-81a8-8b09c106e16f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:32.177809 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1426/2853] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:18:32 2022] POST /identity/v3/users => generated 290 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:18:32.181505 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c823fa6-1218-4b03-81a8-8b09c106e16f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:32.190856 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c823fa6-1218-4b03-81a8-8b09c106e16f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:32.195457 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb28f1a3-fb52-4fc3-a1be-a5b0edddf5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:32.196070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28f1a3-fb52-4fc3-a1be-a5b0edddf5dc admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:32.196387 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28f1a3-fb52-4fc3-a1be-a5b0edddf5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:32.196657 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28f1a3-fb52-4fc3-a1be-a5b0edddf5dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:32.197358 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb28f1a3-fb52-4fc3-a1be-a5b0edddf5dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:32.201018 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb28f1a3-fb52-4fc3-a1be-a5b0edddf5dc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:32.211814 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1427/2854] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:18:32.216117 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb28f1a3-fb52-4fc3-a1be-a5b0edddf5dc admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:32.217078 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6e7c0066-2133-4a9f-bece-c53d15f4bcaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b02048ea85d46889e86bb8d073bf456'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:32.217737 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6e7c0066-2133-4a9f-bece-c53d15f4bcaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b02048ea85d46889e86bb8d073bf456'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:32.226207 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb28f1a3-fb52-4fc3-a1be-a5b0edddf5dc admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:32.231359 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf7e0f-3115-4aaf-a14b-2065b0b8266a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:32.232863 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf7e0f-3115-4aaf-a14b-2065b0b8266a admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:32.233018 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf7e0f-3115-4aaf-a14b-2065b0b8266a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:32.233298 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf7e0f-3115-4aaf-a14b-2065b0b8266a admin admin] PATH_INFO: `/v3/projects/cb1f12ace60745c8aa50637ac4d46354/users/8466880012f040b5bb58143b24060fea/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:32.248863 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf7e0f-3115-4aaf-a14b-2065b0b8266a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb1f12ace60745c8aa50637ac4d46354, user_id=8466880012f040b5bb58143b24060fea, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:32.251563 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf7e0f-3115-4aaf-a14b-2065b0b8266a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:32.297144 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6e7c0066-2133-4a9f-bece-c53d15f4bcaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b02048ea85d46889e86bb8d073bf456'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:32.297847 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6e7c0066-2133-4a9f-bece-c53d15f4bcaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b02048ea85d46889e86bb8d073bf456'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:32.311095 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6e7c0066-2133-4a9f-bece-c53d15f4bcaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b02048ea85d46889e86bb8d073bf456'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:32.323688 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1428/2855] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:18:31 2022] DELETE /identity/v3/users/1b02048ea85d46889e86bb8d073bf456 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:18:32.338372 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1428/2856] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:18:32 2022] PUT /identity/v3/projects/cb1f12ace60745c8aa50637ac4d46354/users/8466880012f040b5bb58143b24060fea/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:18:32.346410 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c0066-2133-4a9f-bece-c53d15f4bcaa admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:32.347810 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6e7c0066-2133-4a9f-bece-c53d15f4bcaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:32.359290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:18:32.420671 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c0066-2133-4a9f-bece-c53d15f4bcaa admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:32.491852 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49b15c-1ce9-4ed8-ac5b-010f1687b3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:32.492571 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49b15c-1ce9-4ed8-ac5b-010f1687b3d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:32.492850 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49b15c-1ce9-4ed8-ac5b-010f1687b3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:32.493124 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49b15c-1ce9-4ed8-ac5b-010f1687b3d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:32.493851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff49b15c-1ce9-4ed8-ac5b-010f1687b3d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:32.495463 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff49b15c-1ce9-4ed8-ac5b-010f1687b3d5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:32.509427 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1429/2857] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:18:32.510631 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf7e0f-3115-4aaf-a14b-2065b0b8266a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:32.521285 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf7e0f-3115-4aaf-a14b-2065b0b8266a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:32.526008 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f1ff6-fad3-4ceb-a90c-67ae8c80683e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:32.526880 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f1ff6-fad3-4ceb-a90c-67ae8c80683e admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:32.527242 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f1ff6-fad3-4ceb-a90c-67ae8c80683e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:32.527438 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f1ff6-fad3-4ceb-a90c-67ae8c80683e admin admin] PATH_INFO: `/v3/projects/cb1f12ace60745c8aa50637ac4d46354/users/8466880012f040b5bb58143b24060fea/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:32.529427 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f1ff6-fad3-4ceb-a90c-67ae8c80683e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb1f12ace60745c8aa50637ac4d46354, user_id=8466880012f040b5bb58143b24060fea, role_id=a9bf1e97c2534a19a5b37cb3e6efbdeb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:32.532070 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f1ff6-fad3-4ceb-a90c-67ae8c80683e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:32.543260 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1429/2858] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:18:32 2022] PUT /identity/v3/projects/cb1f12ace60745c8aa50637ac4d46354/users/8466880012f040b5bb58143b24060fea/roles/a9bf1e97c2534a19a5b37cb3e6efbdeb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:18:32.608447 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08792427-fd12-4642-9d4a-42a2da0f021a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:32.608735 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08792427-fd12-4642-9d4a-42a2da0f021a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:32.609059 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08792427-fd12-4642-9d4a-42a2da0f021a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:32.634752 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f1ff6-fad3-4ceb-a90c-67ae8c80683e admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:32.636556 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-08792427-fd12-4642-9d4a-42a2da0f021a None None] MFA Rules not processed for user `8466880012f040b5bb58143b24060fea`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:18:32.636720 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-040f1ff6-fad3-4ceb-a90c-67ae8c80683e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:32.637887 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-08792427-fd12-4642-9d4a-42a2da0f021a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:32.712711 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1430/2859] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:18:32 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:18:32.715746 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f1ff6-fad3-4ceb-a90c-67ae8c80683e admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:32.720429 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19187eca-ac6e-4d7b-8708-57e211a2a5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:32.721281 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19187eca-ac6e-4d7b-8708-57e211a2a5a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:32.721561 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19187eca-ac6e-4d7b-8708-57e211a2a5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:32.721830 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19187eca-ac6e-4d7b-8708-57e211a2a5a6 admin admin] PATH_INFO: `/v3/projects/eccb032b11e045fdbb42238762eada05` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:32.723012 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19187eca-ac6e-4d7b-8708-57e211a2a5a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eccb032b11e045fdbb42238762eada05)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:32.723306 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08792427-fd12-4642-9d4a-42a2da0f021a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:32.725282 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-08792427-fd12-4642-9d4a-42a2da0f021a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:32.725758 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19187eca-ac6e-4d7b-8708-57e211a2a5a6 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:32.754186 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:18:32.773075 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:18:32.788531 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08792427-fd12-4642-9d4a-42a2da0f021a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:32.793089 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763d95d-94a7-485e-93a7-019314f5e8f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:32.793998 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763d95d-94a7-485e-93a7-019314f5e8f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:32.794281 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763d95d-94a7-485e-93a7-019314f5e8f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:32.794646 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763d95d-94a7-485e-93a7-019314f5e8f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:32.803925 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1763d95d-94a7-485e-93a7-019314f5e8f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:32.805415 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1763d95d-94a7-485e-93a7-019314f5e8f4 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:32.814747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1431/2860] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:18:32 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 20:18:32.857428 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-19187eca-ac6e-4d7b-8708-57e211a2a5a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eccb032b11e045fdbb42238762eada05', 'request_id': 'req-19187eca-ac6e-4d7b-8708-57e211a2a5a6'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:32.863655 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-19187eca-ac6e-4d7b-8708-57e211a2a5a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eccb032b11e045fdbb42238762eada05', 'request_id': 'req-19187eca-ac6e-4d7b-8708-57e211a2a5a6'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:32.869022 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-19187eca-ac6e-4d7b-8708-57e211a2a5a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eccb032b11e045fdbb42238762eada05', 'request_id': 'req-19187eca-ac6e-4d7b-8708-57e211a2a5a6'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:32.949504 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-19187eca-ac6e-4d7b-8708-57e211a2a5a6 admin admin] The token cache is being invalidate because project eccb032b11e045fdbb42238762eada05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:18:32.949789 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-19187eca-ac6e-4d7b-8708-57e211a2a5a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:32.950414 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-19187eca-ac6e-4d7b-8708-57e211a2a5a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:32.951993 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1430/2861] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:18:32 2022] DELETE /identity/v3/projects/eccb032b11e045fdbb42238762eada05 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 20:18:35.445064 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763d95d-94a7-485e-93a7-019314f5e8f4 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:35.455380 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763d95d-94a7-485e-93a7-019314f5e8f4 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:35.511151 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed0c19-9dae-44be-8d52-52bc60d27065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:35.512013 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed0c19-9dae-44be-8d52-52bc60d27065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:35.512375 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed0c19-9dae-44be-8d52-52bc60d27065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:35.512599 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed0c19-9dae-44be-8d52-52bc60d27065 admin admin] PATH_INFO: `/v3/users/2b71c28e501c4de0a947cbf4a05ce89b` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:35.513783 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed0c19-9dae-44be-8d52-52bc60d27065 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b71c28e501c4de0a947cbf4a05ce89b)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:35.515698 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed0c19-9dae-44be-8d52-52bc60d27065 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:35.731977 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-bbed0c19-9dae-44be-8d52-52bc60d27065 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b71c28e501c4de0a947cbf4a05ce89b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:35.732641 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-bbed0c19-9dae-44be-8d52-52bc60d27065 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b71c28e501c4de0a947cbf4a05ce89b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:35.804616 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-bbed0c19-9dae-44be-8d52-52bc60d27065 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b71c28e501c4de0a947cbf4a05ce89b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:35.805276 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-bbed0c19-9dae-44be-8d52-52bc60d27065 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b71c28e501c4de0a947cbf4a05ce89b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:35.818290 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-bbed0c19-9dae-44be-8d52-52bc60d27065 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b71c28e501c4de0a947cbf4a05ce89b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:35.830430 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1432/2862] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:18:35 2022] DELETE /identity/v3/users/2b71c28e501c4de0a947cbf4a05ce89b => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:18:35.835444 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5ff5f-0db5-4ff5-8f60-09be1721e141 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:35.835940 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5ff5f-0db5-4ff5-8f60-09be1721e141 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:35.836061 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5ff5f-0db5-4ff5-8f60-09be1721e141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:35.853932 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c0f5ff5f-0db5-4ff5-8f60-09be1721e141 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:18:35.855125 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c0f5ff5f-0db5-4ff5-8f60-09be1721e141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:35.915058 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1431/2863] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:18:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:18:35.924565 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed0c19-9dae-44be-8d52-52bc60d27065 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:35.926093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bbed0c19-9dae-44be-8d52-52bc60d27065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:35.983580 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed0c19-9dae-44be-8d52-52bc60d27065 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:35.987322 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a65e7-a12f-4960-b3bc-3c03bb6dbf58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:35.987962 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a65e7-a12f-4960-b3bc-3c03bb6dbf58 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:35.988251 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a65e7-a12f-4960-b3bc-3c03bb6dbf58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:35.988554 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a65e7-a12f-4960-b3bc-3c03bb6dbf58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:35.998504 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a65e7-a12f-4960-b3bc-3c03bb6dbf58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:35.999955 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a65e7-a12f-4960-b3bc-3c03bb6dbf58 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:36.009683 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1433/2864] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:18:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:18:36.497783 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5ff5f-0db5-4ff5-8f60-09be1721e141 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:36.499525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c0f5ff5f-0db5-4ff5-8f60-09be1721e141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:36.515323 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5ff5f-0db5-4ff5-8f60-09be1721e141 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:36.520154 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f36137-84b3-40fb-95f5-f562a4edc0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:36.521538 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f36137-84b3-40fb-95f5-f562a4edc0e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:36.521816 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f36137-84b3-40fb-95f5-f562a4edc0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:36.522082 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f36137-84b3-40fb-95f5-f562a4edc0e7 admin admin] PATH_INFO: `/v3/projects/22dcfc78ca3642b683ff276d0caa0eba` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:36.523288 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f36137-84b3-40fb-95f5-f562a4edc0e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22dcfc78ca3642b683ff276d0caa0eba)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:36.525358 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f36137-84b3-40fb-95f5-f562a4edc0e7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:36.663131 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b2f36137-84b3-40fb-95f5-f562a4edc0e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22dcfc78ca3642b683ff276d0caa0eba', 'request_id': 'req-b2f36137-84b3-40fb-95f5-f562a4edc0e7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:36.669266 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b2f36137-84b3-40fb-95f5-f562a4edc0e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22dcfc78ca3642b683ff276d0caa0eba', 'request_id': 'req-b2f36137-84b3-40fb-95f5-f562a4edc0e7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:36.674897 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b2f36137-84b3-40fb-95f5-f562a4edc0e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22dcfc78ca3642b683ff276d0caa0eba', 'request_id': 'req-b2f36137-84b3-40fb-95f5-f562a4edc0e7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:36.759769 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b2f36137-84b3-40fb-95f5-f562a4edc0e7 admin admin] The token cache is being invalidate because project 22dcfc78ca3642b683ff276d0caa0eba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:18:36.760057 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b2f36137-84b3-40fb-95f5-f562a4edc0e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:36.760656 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b2f36137-84b3-40fb-95f5-f562a4edc0e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:36.769405 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1432/2865] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:18:36 2022] DELETE /identity/v3/projects/22dcfc78ca3642b683ff276d0caa0eba => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:18:36.769529 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a65e7-a12f-4960-b3bc-3c03bb6dbf58 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:36.772105 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:18:36.777371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a65e7-a12f-4960-b3bc-3c03bb6dbf58 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:36.834024 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf1906-4a1d-4f07-bf2f-692e4e1742fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:36.834888 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf1906-4a1d-4f07-bf2f-692e4e1742fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:36.835171 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf1906-4a1d-4f07-bf2f-692e4e1742fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:36.835445 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf1906-4a1d-4f07-bf2f-692e4e1742fe admin admin] PATH_INFO: `/v3/users/8466880012f040b5bb58143b24060fea` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:36.836920 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf1906-4a1d-4f07-bf2f-692e4e1742fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=8466880012f040b5bb58143b24060fea)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:36.838848 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf1906-4a1d-4f07-bf2f-692e4e1742fe admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:37.006984 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1fcf1906-4a1d-4f07-bf2f-692e4e1742fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8466880012f040b5bb58143b24060fea'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:37.007648 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1fcf1906-4a1d-4f07-bf2f-692e4e1742fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8466880012f040b5bb58143b24060fea'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:37.079283 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1fcf1906-4a1d-4f07-bf2f-692e4e1742fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8466880012f040b5bb58143b24060fea'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:37.079915 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1fcf1906-4a1d-4f07-bf2f-692e4e1742fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8466880012f040b5bb58143b24060fea'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:37.092252 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1fcf1906-4a1d-4f07-bf2f-692e4e1742fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8466880012f040b5bb58143b24060fea'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:37.104465 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1434/2866] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:18:36 2022] DELETE /identity/v3/users/8466880012f040b5bb58143b24060fea => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:18:37.555503 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f36137-84b3-40fb-95f5-f562a4edc0e7 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:37.556976 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b2f36137-84b3-40fb-95f5-f562a4edc0e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:37.572146 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f36137-84b3-40fb-95f5-f562a4edc0e7 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:37.604212 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:18:37.629873 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911bc9c-624e-41ac-bca0-0abb42bbf1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:37.631242 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911bc9c-624e-41ac-bca0-0abb42bbf1f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:37.631531 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911bc9c-624e-41ac-bca0-0abb42bbf1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:37.631804 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911bc9c-624e-41ac-bca0-0abb42bbf1f2 admin admin] PATH_INFO: `/v3/projects/cb1f12ace60745c8aa50637ac4d46354` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:37.633085 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b911bc9c-624e-41ac-bca0-0abb42bbf1f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb1f12ace60745c8aa50637ac4d46354)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:37.634980 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b911bc9c-624e-41ac-bca0-0abb42bbf1f2 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:37.840238 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b911bc9c-624e-41ac-bca0-0abb42bbf1f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb1f12ace60745c8aa50637ac4d46354', 'request_id': 'req-b911bc9c-624e-41ac-bca0-0abb42bbf1f2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:37.846491 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b911bc9c-624e-41ac-bca0-0abb42bbf1f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb1f12ace60745c8aa50637ac4d46354', 'request_id': 'req-b911bc9c-624e-41ac-bca0-0abb42bbf1f2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:37.852143 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b911bc9c-624e-41ac-bca0-0abb42bbf1f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb1f12ace60745c8aa50637ac4d46354', 'request_id': 'req-b911bc9c-624e-41ac-bca0-0abb42bbf1f2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:37.970517 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b911bc9c-624e-41ac-bca0-0abb42bbf1f2 admin admin] The token cache is being invalidate because project cb1f12ace60745c8aa50637ac4d46354 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:18:37.970827 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b911bc9c-624e-41ac-bca0-0abb42bbf1f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:37.972917 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b911bc9c-624e-41ac-bca0-0abb42bbf1f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:37.973028 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1433/2867] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:18:37 2022] DELETE /identity/v3/projects/cb1f12ace60745c8aa50637ac4d46354 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:18:37.978049 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0d284-ad47-4567-b31a-8382d968d9db None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:37.978527 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0d284-ad47-4567-b31a-8382d968d9db None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:37.978629 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0d284-ad47-4567-b31a-8382d968d9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:37.997161 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-0eb0d284-ad47-4567-b31a-8382d968d9db None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:18:38.001763 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0eb0d284-ad47-4567-b31a-8382d968d9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:38.058358 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1435/2868] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:18:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:18:38.066198 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91f7c8-f0ee-4d0c-8dd3-1cec85534904 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:38.066485 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91f7c8-f0ee-4d0c-8dd3-1cec85534904 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:38.066763 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de91f7c8-f0ee-4d0c-8dd3-1cec85534904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:38.084521 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-de91f7c8-f0ee-4d0c-8dd3-1cec85534904 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:18:38.086085 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-de91f7c8-f0ee-4d0c-8dd3-1cec85534904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:38.093855 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1434/2869] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:18:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:18:38.098005 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0d284-ad47-4567-b31a-8382d968d9db None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:38.108172 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0d284-ad47-4567-b31a-8382d968d9db None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:38.113076 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6d676-ea82-40a9-a444-7d40034d80ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:38.116547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6d676-ea82-40a9-a444-7d40034d80ed admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:38.116547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6d676-ea82-40a9-a444-7d40034d80ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:38.116547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6d676-ea82-40a9-a444-7d40034d80ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:38.116547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad6d676-ea82-40a9-a444-7d40034d80ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 20:18:38.116547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad6d676-ea82-40a9-a444-7d40034d80ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:38.117541 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad6d676-ea82-40a9-a444-7d40034d80ed admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:38.129159 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1436/2870] 10.222.0.44 () {58 vars in 1152 bytes} [Thu Jun 23 20:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 20:18:38.133633 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91f7c8-f0ee-4d0c-8dd3-1cec85534904 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:38.143106 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de91f7c8-f0ee-4d0c-8dd3-1cec85534904 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:38.147847 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec2234-b749-438f-9863-a8338778481f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:38.148510 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec2234-b749-438f-9863-a8338778481f admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:38.148789 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec2234-b749-438f-9863-a8338778481f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:38.149056 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec2234-b749-438f-9863-a8338778481f admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:38.149741 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec2234-b749-438f-9863-a8338778481f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:38.151732 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec2234-b749-438f-9863-a8338778481f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:38.228538 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1435/2871] 10.222.0.44 () {60 vars in 1152 bytes} [Thu Jun 23 20:18:38 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:18:38.232619 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6d676-ea82-40a9-a444-7d40034d80ed admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:38.243878 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6d676-ea82-40a9-a444-7d40034d80ed admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:38.248412 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:18:38.299593 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3532894-c378-4b85-af42-708dcdd272b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:38.300335 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3532894-c378-4b85-af42-708dcdd272b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:38.300652 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3532894-c378-4b85-af42-708dcdd272b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:38.300924 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3532894-c378-4b85-af42-708dcdd272b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:38.301608 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3532894-c378-4b85-af42-708dcdd272b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:38.304876 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3532894-c378-4b85-af42-708dcdd272b5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:38.383265 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1437/2872] 10.222.0.44 () {60 vars in 1143 bytes} [Thu Jun 23 20:18:38 2022] POST /identity/v3/users => generated 293 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 20:18:38.387465 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec2234-b749-438f-9863-a8338778481f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:38.400118 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec2234-b749-438f-9863-a8338778481f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:38.404831 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939daccd-6cb1-483d-8674-e1a4bf6e08a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:38.405497 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939daccd-6cb1-483d-8674-e1a4bf6e08a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:38.405780 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939daccd-6cb1-483d-8674-e1a4bf6e08a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:38.406054 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939daccd-6cb1-483d-8674-e1a4bf6e08a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:38.406798 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939daccd-6cb1-483d-8674-e1a4bf6e08a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:38.408416 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939daccd-6cb1-483d-8674-e1a4bf6e08a4 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:38.419298 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1436/2873] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 20:18:38.423357 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3532894-c378-4b85-af42-708dcdd272b5 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:38.434080 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3532894-c378-4b85-af42-708dcdd272b5 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:38.439138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef74e4-1917-4b2d-ac26-752c342faa91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:38.439805 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef74e4-1917-4b2d-ac26-752c342faa91 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:38.440094 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef74e4-1917-4b2d-ac26-752c342faa91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:38.440408 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef74e4-1917-4b2d-ac26-752c342faa91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:38.441148 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef74e4-1917-4b2d-ac26-752c342faa91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:38.442807 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef74e4-1917-4b2d-ac26-752c342faa91 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:38.453114 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1438/2874] 10.222.0.44 () {58 vars in 1121 bytes} [Thu Jun 23 20:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 20:18:38.457229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939daccd-6cb1-483d-8674-e1a4bf6e08a4 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:38.468461 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939daccd-6cb1-483d-8674-e1a4bf6e08a4 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:38.473229 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30563fdd-7ae9-472c-a668-567092a4d700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:38.474693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30563fdd-7ae9-472c-a668-567092a4d700 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:38.474971 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30563fdd-7ae9-472c-a668-567092a4d700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:38.475263 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30563fdd-7ae9-472c-a668-567092a4d700 admin admin] PATH_INFO: `/v3/projects/03fdcb4bcdb741d2b949c617bcaee595/users/cf2f8f862b8b4205ac20d73cb230a719/roles/860491bfbb574772a7a213d6a25962e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:38.490739 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30563fdd-7ae9-472c-a668-567092a4d700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03fdcb4bcdb741d2b949c617bcaee595, user_id=cf2f8f862b8b4205ac20d73cb230a719, role_id=860491bfbb574772a7a213d6a25962e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:38.493395 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30563fdd-7ae9-472c-a668-567092a4d700 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:38.553449 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1437/2875] 10.222.0.44 () {60 vars in 1482 bytes} [Thu Jun 23 20:18:38 2022] PUT /identity/v3/projects/03fdcb4bcdb741d2b949c617bcaee595/users/cf2f8f862b8b4205ac20d73cb230a719/roles/860491bfbb574772a7a213d6a25962e0 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:18:38.563494 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65fd3f1-dcaa-4daa-9aa5-ad409ce95c71 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:38.563801 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65fd3f1-dcaa-4daa-9aa5-ad409ce95c71 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:38.564134 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65fd3f1-dcaa-4daa-9aa5-ad409ce95c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:38.590819 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b65fd3f1-dcaa-4daa-9aa5-ad409ce95c71 None None] MFA Rules not processed for user `cf2f8f862b8b4205ac20d73cb230a719`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:18:38.592080 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b65fd3f1-dcaa-4daa-9aa5-ad409ce95c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:38.655547 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1439/2876] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:18:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:18:38.667109 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30563fdd-7ae9-472c-a668-567092a4d700 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:38.668910 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-30563fdd-7ae9-472c-a668-567092a4d700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:38.728195 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30563fdd-7ae9-472c-a668-567092a4d700 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:38.731929 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff331f2-cc46-48f3-9bf6-8b59c22bce18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21ae3b8971b44d928e9f6c0539858636', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:38.732599 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff331f2-cc46-48f3-9bf6-8b59c22bce18 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:38.732878 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff331f2-cc46-48f3-9bf6-8b59c22bce18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:38.733148 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff331f2-cc46-48f3-9bf6-8b59c22bce18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:38.741567 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff331f2-cc46-48f3-9bf6-8b59c22bce18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:38.742957 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff331f2-cc46-48f3-9bf6-8b59c22bce18 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:38.751581 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1438/2877] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:18:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:18:41.240553 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65fd3f1-dcaa-4daa-9aa5-ad409ce95c71 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:41.245552 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b65fd3f1-dcaa-4daa-9aa5-ad409ce95c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:41.314063 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65fd3f1-dcaa-4daa-9aa5-ad409ce95c71 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:41.320132 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e48efe-e1a4-4e68-9cda-1f164a5f7e88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:41.320803 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e48efe-e1a4-4e68-9cda-1f164a5f7e88 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:41.321083 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e48efe-e1a4-4e68-9cda-1f164a5f7e88 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:41.321350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e48efe-e1a4-4e68-9cda-1f164a5f7e88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:41.323138 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c7e48efe-e1a4-4e68-9cda-1f164a5f7e88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:41.336843 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e48efe-e1a4-4e68-9cda-1f164a5f7e88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:41.343032 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e48efe-e1a4-4e68-9cda-1f164a5f7e88 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:41.349578 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1440/2878] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:18:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:18:45.696635 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff331f2-cc46-48f3-9bf6-8b59c22bce18 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:45.698019 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-cff331f2-cc46-48f3-9bf6-8b59c22bce18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:45.703390 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:18:45.757346 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff331f2-cc46-48f3-9bf6-8b59c22bce18 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:45.760998 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bdda1b-d9a0-427b-91b3-57b10e15f452 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:45.761644 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdda1b-d9a0-427b-91b3-57b10e15f452 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:45.761986 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdda1b-d9a0-427b-91b3-57b10e15f452 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:45.762232 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdda1b-d9a0-427b-91b3-57b10e15f452 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:45.764033 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c2bdda1b-d9a0-427b-91b3-57b10e15f452 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:45.827218 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bdda1b-d9a0-427b-91b3-57b10e15f452 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:45.828697 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bdda1b-d9a0-427b-91b3-57b10e15f452 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:45.838846 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1439/2879] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:18:45 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:18:47.696444 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e48efe-e1a4-4e68-9cda-1f164a5f7e88 service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:47.702348 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:18:47.709176 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e48efe-e1a4-4e68-9cda-1f164a5f7e88 service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:47.716922 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c03dae-c11b-467f-ad7b-0cb7c07a50ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:47.717567 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c03dae-c11b-467f-ad7b-0cb7c07a50ad service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:47.717851 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c03dae-c11b-467f-ad7b-0cb7c07a50ad service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:47.718128 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c03dae-c11b-467f-ad7b-0cb7c07a50ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:47.720019 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e8c03dae-c11b-467f-ad7b-0cb7c07a50ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:47.735718 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c03dae-c11b-467f-ad7b-0cb7c07a50ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:47.737371 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c03dae-c11b-467f-ad7b-0cb7c07a50ad service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:47.749727 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1441/2880] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:18:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:18:48.556655 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bdda1b-d9a0-427b-91b3-57b10e15f452 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:48.619159 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bdda1b-d9a0-427b-91b3-57b10e15f452 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:48.623700 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ed3c1-607e-4dcb-84f4-3eb95fb67a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:48.624602 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ed3c1-607e-4dcb-84f4-3eb95fb67a05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:48.624880 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ed3c1-607e-4dcb-84f4-3eb95fb67a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:48.625152 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ed3c1-607e-4dcb-84f4-3eb95fb67a05 admin admin] PATH_INFO: `/v3/users/cf2f8f862b8b4205ac20d73cb230a719` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:48.626310 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ed3c1-607e-4dcb-84f4-3eb95fb67a05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf2f8f862b8b4205ac20d73cb230a719)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:48.628211 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ed3c1-607e-4dcb-84f4-3eb95fb67a05 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:48.818373 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6f2ed3c1-607e-4dcb-84f4-3eb95fb67a05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf2f8f862b8b4205ac20d73cb230a719'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:48.819024 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6f2ed3c1-607e-4dcb-84f4-3eb95fb67a05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf2f8f862b8b4205ac20d73cb230a719'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:48.824080 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:18:48.830786 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6f2ed3c1-607e-4dcb-84f4-3eb95fb67a05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf2f8f862b8b4205ac20d73cb230a719'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:48.837744 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6f2ed3c1-607e-4dcb-84f4-3eb95fb67a05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf2f8f862b8b4205ac20d73cb230a719'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:48.885360 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6f2ed3c1-607e-4dcb-84f4-3eb95fb67a05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf2f8f862b8b4205ac20d73cb230a719'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:48.891368 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1440/2881] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:18:48 2022] DELETE /identity/v3/users/cf2f8f862b8b4205ac20d73cb230a719 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:18:48.895938 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3bfe6-e375-4c78-b169-630680645c58 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:48.896438 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3bfe6-e375-4c78-b169-630680645c58 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:48.896610 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3bfe6-e375-4c78-b169-630680645c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:48.915227 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-01b3bfe6-e375-4c78-b169-630680645c58 None None] MFA Rules not processed for user `5060168fc25744178d345904afb6c333`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:18:48.915496 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-01b3bfe6-e375-4c78-b169-630680645c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:48.972257 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1442/2882] 10.222.0.44 () {58 vars in 957 bytes} [Thu Jun 23 20:18:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:18:48.982583 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ed3c1-607e-4dcb-84f4-3eb95fb67a05 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:48.984002 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6f2ed3c1-607e-4dcb-84f4-3eb95fb67a05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:49.040098 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ed3c1-607e-4dcb-84f4-3eb95fb67a05 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:49.043659 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a3a39-a853-41a6-a55e-359ebe9a0103 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:49.044315 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a3a39-a853-41a6-a55e-359ebe9a0103 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:49.044600 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a3a39-a853-41a6-a55e-359ebe9a0103 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:49.044874 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a3a39-a853-41a6-a55e-359ebe9a0103 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:49.053807 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a3a39-a853-41a6-a55e-359ebe9a0103 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:49.055228 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a3a39-a853-41a6-a55e-359ebe9a0103 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:49.064693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1441/2883] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:18:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:18:49.412927 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3bfe6-e375-4c78-b169-630680645c58 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:18:49.414352 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-01b3bfe6-e375-4c78-b169-630680645c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:18:49.427358 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3bfe6-e375-4c78-b169-630680645c58 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:18:49.432722 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e67df-0c67-40e7-9f72-70182a4318ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:18:49.433659 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e67df-0c67-40e7-9f72-70182a4318ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:18:49.433947 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e67df-0c67-40e7-9f72-70182a4318ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:18:49.434221 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e67df-0c67-40e7-9f72-70182a4318ac admin admin] PATH_INFO: `/v3/projects/03fdcb4bcdb741d2b949c617bcaee595` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:18:49.435448 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e67df-0c67-40e7-9f72-70182a4318ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=03fdcb4bcdb741d2b949c617bcaee595)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:18:49.437487 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e67df-0c67-40e7-9f72-70182a4318ac admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:18:49.565285 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7a2e67df-0c67-40e7-9f72-70182a4318ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03fdcb4bcdb741d2b949c617bcaee595', 'request_id': 'req-7a2e67df-0c67-40e7-9f72-70182a4318ac'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:49.650582 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7a2e67df-0c67-40e7-9f72-70182a4318ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03fdcb4bcdb741d2b949c617bcaee595', 'request_id': 'req-7a2e67df-0c67-40e7-9f72-70182a4318ac'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:49.656589 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7a2e67df-0c67-40e7-9f72-70182a4318ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03fdcb4bcdb741d2b949c617bcaee595', 'request_id': 'req-7a2e67df-0c67-40e7-9f72-70182a4318ac'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:49.667642 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7a2e67df-0c67-40e7-9f72-70182a4318ac admin admin] The token cache is being invalidate because project 03fdcb4bcdb741d2b949c617bcaee595 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:18:49.667933 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7a2e67df-0c67-40e7-9f72-70182a4318ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:49.668580 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7a2e67df-0c67-40e7-9f72-70182a4318ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:18:49.670140 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1443/2884] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:18:49 2022] DELETE /identity/v3/projects/03fdcb4bcdb741d2b949c617bcaee595 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:18:59.060432 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=36,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:18:59.664690 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=59,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:20:57.201280 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a3a39-a853-41a6-a55e-359ebe9a0103 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:20:57.202701 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1b4a3a39-a853-41a6-a55e-359ebe9a0103 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:20:57.342243 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a3a39-a853-41a6-a55e-359ebe9a0103 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:20:57.401946 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a21ed-8b02-4a12-9746-1e696680ea4a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9715d5deeccd419b97cb0bd39e17982f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:20:57.402624 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a21ed-8b02-4a12-9746-1e696680ea4a service placement] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:20:57.402910 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a21ed-8b02-4a12-9746-1e696680ea4a service placement] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:20:57.403188 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a21ed-8b02-4a12-9746-1e696680ea4a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:20:57.405564 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-213a21ed-8b02-4a12-9746-1e696680ea4a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:20:57.416532 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a21ed-8b02-4a12-9746-1e696680ea4a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:20:57.418105 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a21ed-8b02-4a12-9746-1e696680ea4a service placement] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:20:57.427774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1442/2885] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:20:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:20:58.807747 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfc1ad-e291-4687-947c-fd0c3c8304d9 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:20:58.808347 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfc1ad-e291-4687-947c-fd0c3c8304d9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:20:58.808347 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfc1ad-e291-4687-947c-fd0c3c8304d9 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:20:58.809745 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1444/2886] 10.222.0.44 () {54 vars in 921 bytes} [Thu Jun 23 20:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:20:58.815712 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618491d-9bc2-42f6-a297-74ea384caa20 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:20:58.819156 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618491d-9bc2-42f6-a297-74ea384caa20 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:20:58.819156 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5618491d-9bc2-42f6-a297-74ea384caa20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:20:58.848416 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:20:58.892798 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5618491d-9bc2-42f6-a297-74ea384caa20 None None] MFA Rules not processed for user `332fbbefba114da4a01fdd496fff1e31`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:20:58.894847 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5618491d-9bc2-42f6-a297-74ea384caa20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:20:58.965145 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1443/2887] 10.222.0.44 () {58 vars in 1018 bytes} [Thu Jun 23 20:20:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 20:20:58.977546 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dfc1ad-e291-4687-947c-fd0c3c8304d9 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:20:58.979421 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-59dfc1ad-e291-4687-947c-fd0c3c8304d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:20:58.993146 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dfc1ad-e291-4687-947c-fd0c3c8304d9 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:20:58.996808 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484c92e1-342a-4193-8775-1a702ae2d016 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:20:58.997437 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c92e1-342a-4193-8775-1a702ae2d016 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:20:58.997719 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c92e1-342a-4193-8775-1a702ae2d016 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:20:58.997992 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484c92e1-342a-4193-8775-1a702ae2d016 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:20:59.005221 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484c92e1-342a-4193-8775-1a702ae2d016 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:20:59.006687 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484c92e1-342a-4193-8775-1a702ae2d016 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:20:59.014577 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1445/2888] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:20:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:20:59.359259 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77419db-96bd-4559-986d-e5255ffe5f1a None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:20:59.359551 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77419db-96bd-4559-986d-e5255ffe5f1a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:20:59.359821 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77419db-96bd-4559-986d-e5255ffe5f1a None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:20:59.361326 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1444/2889] 10.222.0.44 () {54 vars in 921 bytes} [Thu Jun 23 20:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 20:20:59.366435 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be2419a-29e2-4c7d-9d4d-74375b6cfd8d None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:20:59.366726 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be2419a-29e2-4c7d-9d4d-74375b6cfd8d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:20:59.367072 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be2419a-29e2-4c7d-9d4d-74375b6cfd8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:20:59.415626 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3be2419a-29e2-4c7d-9d4d-74375b6cfd8d None None] MFA Rules not processed for user `a63d1d902c024c97a38f7d92f73814d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 20:20:59.416840 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3be2419a-29e2-4c7d-9d4d-74375b6cfd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:20:59.500581 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1446/2890] 10.222.0.44 () {58 vars in 1018 bytes} [Thu Jun 23 20:20:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 20:20:59.511546 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77419db-96bd-4559-986d-e5255ffe5f1a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:20:59.523037 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77419db-96bd-4559-986d-e5255ffe5f1a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:20:59.526513 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33883d-f03b-4cab-99c1-8f91fc4f51b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:20:59.527130 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33883d-f03b-4cab-99c1-8f91fc4f51b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:20:59.527410 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33883d-f03b-4cab-99c1-8f91fc4f51b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:20:59.527681 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33883d-f03b-4cab-99c1-8f91fc4f51b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:20:59.537547 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33883d-f03b-4cab-99c1-8f91fc4f51b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:20:59.538984 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33883d-f03b-4cab-99c1-8f91fc4f51b6 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:20:59.549374 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1445/2891] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:20:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:21:09.497669 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=55 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:21:09.544861 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=41 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:22:30.785889 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be2419a-29e2-4c7d-9d4d-74375b6cfd8d None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:22:30.796482 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be2419a-29e2-4c7d-9d4d-74375b6cfd8d None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:22:30.800407 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89099d90-b374-491e-94d1-80a8cb677bf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:22:30.801049 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89099d90-b374-491e-94d1-80a8cb677bf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:22:30.801347 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89099d90-b374-491e-94d1-80a8cb677bf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:22:30.801734 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89099d90-b374-491e-94d1-80a8cb677bf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:22:30.803486 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-89099d90-b374-491e-94d1-80a8cb677bf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:22:30.817775 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89099d90-b374-491e-94d1-80a8cb677bf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:22:30.819200 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89099d90-b374-491e-94d1-80a8cb677bf9 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:22:30.831107 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1447/2892] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:22:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:22:32.010342 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33883d-f03b-4cab-99c1-8f91fc4f51b6 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:22:32.019082 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33883d-f03b-4cab-99c1-8f91fc4f51b6 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:22:32.022752 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa3a803-1736-4be6-8162-c3001274f41a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332fbbefba114da4a01fdd496fff1e31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:22:32.023483 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3a803-1736-4be6-8162-c3001274f41a service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:22:32.023774 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3a803-1736-4be6-8162-c3001274f41a service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:22:32.024052 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3a803-1736-4be6-8162-c3001274f41a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:22:32.025846 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-eaa3a803-1736-4be6-8162-c3001274f41a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:22:32.040442 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa3a803-1736-4be6-8162-c3001274f41a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:22:32.044525 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa3a803-1736-4be6-8162-c3001274f41a service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:22:32.053266 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1446/2893] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:22:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:22:32.959274 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89099d90-b374-491e-94d1-80a8cb677bf9 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:22:32.960704 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-89099d90-b374-491e-94d1-80a8cb677bf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:22:32.974531 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89099d90-b374-491e-94d1-80a8cb677bf9 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:22:32.980145 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c69d1c-cd82-42bd-9d9f-f34cdcc2336c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a63d1d902c024c97a38f7d92f73814d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:22:32.980808 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c69d1c-cd82-42bd-9d9f-f34cdcc2336c service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:22:32.981093 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c69d1c-cd82-42bd-9d9f-f34cdcc2336c service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:22:32.981370 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c69d1c-cd82-42bd-9d9f-f34cdcc2336c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:22:32.983107 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c7c69d1c-cd82-42bd-9d9f-f34cdcc2336c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:22:32.995862 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c69d1c-cd82-42bd-9d9f-f34cdcc2336c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:22:32.997490 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c69d1c-cd82-42bd-9d9f-f34cdcc2336c service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:22:33.009171 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1448/2894] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:22:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:22:39.411686 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa3a803-1736-4be6-8162-c3001274f41a service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:22:39.413327 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-eaa3a803-1736-4be6-8162-c3001274f41a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:22:39.430284 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa3a803-1736-4be6-8162-c3001274f41a service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:22:39.439493 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad9d530-cb3e-4ba8-823d-e64109bd2577 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9715d5deeccd419b97cb0bd39e17982f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:22:39.440166 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad9d530-cb3e-4ba8-823d-e64109bd2577 service placement] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:22:39.440490 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad9d530-cb3e-4ba8-823d-e64109bd2577 service placement] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:22:39.440772 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad9d530-cb3e-4ba8-823d-e64109bd2577 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:22:39.442645 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7ad9d530-cb3e-4ba8-823d-e64109bd2577 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:22:39.454813 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad9d530-cb3e-4ba8-823d-e64109bd2577 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:22:39.456411 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad9d530-cb3e-4ba8-823d-e64109bd2577 service placement] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:22:39.467581 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1447/2895] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:22:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:22:43.003546 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=12 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:22:43.115185 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c69d1c-cd82-42bd-9d9f-f34cdcc2336c service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:22:43.116918 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c7c69d1c-cd82-42bd-9d9f-f34cdcc2336c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:22:43.220642 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c69d1c-cd82-42bd-9d9f-f34cdcc2336c service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:22:43.225350 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7bff54-26a1-4fbc-b9d7-c8e5f6aa0694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:22:43.226234 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bff54-26a1-4fbc-b9d7-c8e5f6aa0694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:22:43.226519 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bff54-26a1-4fbc-b9d7-c8e5f6aa0694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:22:43.226790 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bff54-26a1-4fbc-b9d7-c8e5f6aa0694 admin admin] PATH_INFO: `/v3/users/3fa08242da514c0d9fa719830dcfc760` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:22:43.228007 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7bff54-26a1-4fbc-b9d7-c8e5f6aa0694 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fa08242da514c0d9fa719830dcfc760)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:22:43.230085 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7bff54-26a1-4fbc-b9d7-c8e5f6aa0694 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:22:43.400097 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ba7bff54-26a1-4fbc-b9d7-c8e5f6aa0694 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fa08242da514c0d9fa719830dcfc760'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:22:43.400890 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ba7bff54-26a1-4fbc-b9d7-c8e5f6aa0694 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fa08242da514c0d9fa719830dcfc760'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:22:43.488981 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ba7bff54-26a1-4fbc-b9d7-c8e5f6aa0694 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fa08242da514c0d9fa719830dcfc760'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:22:43.489591 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ba7bff54-26a1-4fbc-b9d7-c8e5f6aa0694 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fa08242da514c0d9fa719830dcfc760'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:22:43.502809 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ba7bff54-26a1-4fbc-b9d7-c8e5f6aa0694 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fa08242da514c0d9fa719830dcfc760'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:22:43.515378 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1449/2896] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:22:43 2022] DELETE /identity/v3/users/3fa08242da514c0d9fa719830dcfc760 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:22:43.806693 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad9d530-cb3e-4ba8-823d-e64109bd2577 service placement] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:22:43.817970 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad9d530-cb3e-4ba8-823d-e64109bd2577 service placement] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:22:43.883826 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8cd35-6c66-4578-9bc0-6f7a29a08980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:22:43.884731 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8cd35-6c66-4578-9bc0-6f7a29a08980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:22:43.885014 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8cd35-6c66-4578-9bc0-6f7a29a08980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:22:43.885289 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8cd35-6c66-4578-9bc0-6f7a29a08980 admin admin] PATH_INFO: `/v3/projects/47ee9522c6214257b59f3a350220dc0b` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:22:43.886466 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c8cd35-6c66-4578-9bc0-6f7a29a08980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47ee9522c6214257b59f3a350220dc0b)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:22:43.888893 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c8cd35-6c66-4578-9bc0-6f7a29a08980 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:22:43.987644 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-47c8cd35-6c66-4578-9bc0-6f7a29a08980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47ee9522c6214257b59f3a350220dc0b', 'request_id': 'req-47c8cd35-6c66-4578-9bc0-6f7a29a08980'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:22:43.993816 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-47c8cd35-6c66-4578-9bc0-6f7a29a08980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47ee9522c6214257b59f3a350220dc0b', 'request_id': 'req-47c8cd35-6c66-4578-9bc0-6f7a29a08980'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:22:43.999845 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-47c8cd35-6c66-4578-9bc0-6f7a29a08980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47ee9522c6214257b59f3a350220dc0b', 'request_id': 'req-47c8cd35-6c66-4578-9bc0-6f7a29a08980'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:22:44.077206 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-47c8cd35-6c66-4578-9bc0-6f7a29a08980 admin admin] The token cache is being invalidate because project 47ee9522c6214257b59f3a350220dc0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:22:44.077496 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-47c8cd35-6c66-4578-9bc0-6f7a29a08980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:22:44.078116 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-47c8cd35-6c66-4578-9bc0-6f7a29a08980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:22:44.079765 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1448/2897] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:22:43 2022] DELETE /identity/v3/projects/47ee9522c6214257b59f3a350220dc0b => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:22:44.083732 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7bff54-26a1-4fbc-b9d7-c8e5f6aa0694 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:22:44.085220 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ba7bff54-26a1-4fbc-b9d7-c8e5f6aa0694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:22:44.098503 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7bff54-26a1-4fbc-b9d7-c8e5f6aa0694 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:22:44.144806 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:22:44.154141 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a69a589-c7e5-48d2-b6df-25e32204b598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:22:44.155010 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69a589-c7e5-48d2-b6df-25e32204b598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:22:44.155323 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69a589-c7e5-48d2-b6df-25e32204b598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:22:44.155603 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a69a589-c7e5-48d2-b6df-25e32204b598 admin admin] PATH_INFO: `/v3/users/2ebdcce97d7c49aa8edaf8d2fb069962` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:22:44.156806 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a69a589-c7e5-48d2-b6df-25e32204b598 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ebdcce97d7c49aa8edaf8d2fb069962)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:22:44.158734 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a69a589-c7e5-48d2-b6df-25e32204b598 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:22:44.378531 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8a69a589-c7e5-48d2-b6df-25e32204b598 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ebdcce97d7c49aa8edaf8d2fb069962'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:22:44.379118 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8a69a589-c7e5-48d2-b6df-25e32204b598 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ebdcce97d7c49aa8edaf8d2fb069962'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:22:44.444860 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8a69a589-c7e5-48d2-b6df-25e32204b598 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ebdcce97d7c49aa8edaf8d2fb069962'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:22:44.445491 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8a69a589-c7e5-48d2-b6df-25e32204b598 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ebdcce97d7c49aa8edaf8d2fb069962'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:22:44.458771 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8a69a589-c7e5-48d2-b6df-25e32204b598 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ebdcce97d7c49aa8edaf8d2fb069962'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:22:44.475340 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1450/2898] 10.222.0.44 () {58 vars in 1223 bytes} [Thu Jun 23 20:22:44 2022] DELETE /identity/v3/users/2ebdcce97d7c49aa8edaf8d2fb069962 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:22:44.727250 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8cd35-6c66-4578-9bc0-6f7a29a08980 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:22:44.730462 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-47c8cd35-6c66-4578-9bc0-6f7a29a08980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:22:44.742105 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8cd35-6c66-4578-9bc0-6f7a29a08980 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:22:44.804423 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d536d9-e461-4292-9836-bacc950d7bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5060168fc25744178d345904afb6c333', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e3c8189cd37644e9a49e850123b33202', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:22:44.805285 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d536d9-e461-4292-9836-bacc950d7bae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:22:44.805564 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d536d9-e461-4292-9836-bacc950d7bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:22:44.805839 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d536d9-e461-4292-9836-bacc950d7bae admin admin] PATH_INFO: `/v3/projects/4b19b714218d4bea9818c0162a66b1b4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:22:44.807029 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d536d9-e461-4292-9836-bacc950d7bae admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b19b714218d4bea9818c0162a66b1b4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:22:44.808969 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d536d9-e461-4292-9836-bacc950d7bae admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:22:44.815869 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:22:44.940460 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-95d536d9-e461-4292-9836-bacc950d7bae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b19b714218d4bea9818c0162a66b1b4', 'request_id': 'req-95d536d9-e461-4292-9836-bacc950d7bae'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:22:44.946475 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-95d536d9-e461-4292-9836-bacc950d7bae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b19b714218d4bea9818c0162a66b1b4', 'request_id': 'req-95d536d9-e461-4292-9836-bacc950d7bae'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:22:44.951822 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-95d536d9-e461-4292-9836-bacc950d7bae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b19b714218d4bea9818c0162a66b1b4', 'request_id': 'req-95d536d9-e461-4292-9836-bacc950d7bae'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:22:45.036397 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-95d536d9-e461-4292-9836-bacc950d7bae admin admin] The token cache is being invalidate because project 4b19b714218d4bea9818c0162a66b1b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 20:22:45.036689 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-95d536d9-e461-4292-9836-bacc950d7bae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:22:45.037316 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-95d536d9-e461-4292-9836-bacc950d7bae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 20:22:45.038859 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1449/2899] 10.222.0.44 () {58 vars in 1232 bytes} [Thu Jun 23 20:22:44 2022] DELETE /identity/v3/projects/4b19b714218d4bea9818c0162a66b1b4 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 20:22:54.471570 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=30,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:22:55.077833 dv-cinder-847274-2-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 20:26:21.637564 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a69a589-c7e5-48d2-b6df-25e32204b598 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 20:26:21.638978 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8a69a589-c7e5-48d2-b6df-25e32204b598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:26:21.705523 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a69a589-c7e5-48d2-b6df-25e32204b598 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 20:26:21.762698 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76573736-3bc1-4451-978b-52687146788e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9715d5deeccd419b97cb0bd39e17982f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '155180ba63c446ca889a6eb3f94cc31a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 20:26:21.763366 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76573736-3bc1-4451-978b-52687146788e service placement] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 20:26:21.763615 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76573736-3bc1-4451-978b-52687146788e service placement] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 20:26:21.763884 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76573736-3bc1-4451-978b-52687146788e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 20:26:21.765654 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-76573736-3bc1-4451-978b-52687146788e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 20:26:21.777071 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76573736-3bc1-4451-978b-52687146788e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 20:26:21.778649 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76573736-3bc1-4451-978b-52687146788e service placement] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 20:26:21.788980 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1451/2900] 10.222.0.44 () {60 vars in 1365 bytes} [Thu Jun 23 20:26:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 20:26:31.784180 dv-cinder-847274-2-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=58 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}